I want to write about how we got there, because the freedom only makes sense next to what came before it. I wouldn't say my experience applies to everyone raised Mormon. Plenty of people grow up in the church and have a very different story. This one is mine.

I also want to be clear about something before I start. There are thousands of wonderful, kind people in that church, genuinely some of the best people I've worked with. I do think kids growing up in it have a harder life than they should because of the religion. But I don't hate anyone in it, or in any religion.

I was raised in a home that was not only overly strict but controlling. We couldn't even hang out with kids who drank coffee. Coffee. That's insane. It says a lot that a friendship could be ruled out by what was in someone's cup, before anyone asked what kind of person they were.

Arguments were worse. Every discussion that got heated and turned into an argument was treated as the devil working through us and taking over our home, and my father had to banish the devil. It was overly dramatic and emotional, and looking back, it was all to maintain control.

When disagreement is the devil, there's no such thing as a fair argument. The person who pushed back wasn't just wrong; they were the thing the house had to be cleansed of. I don't think you learn how to work through conflict in a home like that. You learn that the safest thing is not to have it.

Not all of that is tied to the religion. Controlling parents exist everywhere, in every faith and in none. But the religion supports it. In our house, scripture backed it up: the man owned and controlled everything. When a father's authority comes with God's name on it, questioning him stops being disagreement and becomes rebellion.

Years later, my wife and I had two young kids of our own, and we had to decide what we wanted their childhood to be. We realized we didn't want them indoctrinated into a life full of catch-22s.

I mean the phrases you hear over and over: "Doubt your doubts." "Just have faith; not all questions need answers." They sound humble, even comforting. But that is the root of the problem, isn't it? The same system that tells you not every question needs an answer also wants you completely certain about what this life is for and what comes next. You're expected to be sure of the biggest answers and incurious about the questions underneath them. And if you do doubt, the doubt is what's wrong with you.

Voltaire put it better than I can:

Doubt is not a pleasant condition, but certainty is an absurd one.

Voltaire, in a letter to Frederick William, Prince of Prussia, in 1770 (translated by S. G. Tallentyre).

Doubt isn't comfortable. Voltaire admits that much. But it's honest. It leaves room to learn something, to be wrong and to change your mind. Certainty closes that room, and when certainty is required, the people asking questions become the problem. That wasn't something we wanted to hand to our kids.

Then COVID-19 hit, and we finally got a long break from drinking the Kool-Aid.

Before that, so much of our time had gone to church on Sundays and extra events during the week that time with our family was low. Now we didn't stress about Sundays or about finding time for the kids. We were just together. The pressure was gone, and freedom was present in everything.

One Sunday morning stands out. A couple of weeks had gone by, and it hit me: we could just spend the next few hours building a Lego set, my son and I. No pressure. We made some breakfast and wasted the morning away. By the standards I grew up with, that's exactly what it was, a Sunday spent on nothing. But it didn't feel like nothing. We bonded, and we enjoyed that time together more than ever before.

I don't think we understood how much the schedule had been carrying until it stopped. Some of the weight was the hours themselves. More of it was the feeling that our time was already spoken for, and that any hour we kept for ourselves needed a reason.

The bigger realization came later, talking with people who weren't raised in a religion like mine. What felt like a revelation to me was just ordinary life to them.

That is normal. Freedom is normal. Spending your time how you want is normal. You don't have to be doing something for someone all the time. You won't go to hell if you don't.

Written out, it seems almost too simple to need saying. But when you've been taught that every hour should go to doing something for someone, and that eternity depends on it, it does need saying. Sometimes more than once.

Some time later, I turned to my wife and said, "Does this really make sense to you? Is this really who we are? Is this the life we want for our kids? I think I'm done. I don't think I can stay in this church anymore."

Her answer was, "It's about time. I've been done for over a year, but I just kept pushing through it. How can I raise my kids knowing what could happen to one of them if they happened to be gay? Or pressure them to be active and a part of something that even I'm not convinced of? There is no proof. Only ample evidence against it."

From that moment on, a new door opened for us. We could be open with each other and talk about anything: think through problems and solutions, or just vent. There was no limit to it anymore. It was like we were finally one.

A couple of years on, we've really enjoyed being separated from the religion. It hasn't all been easy. My wife's family has been very supportive. Mine is more mixed. I come from a big family, eleven of us, and there wasn't one reaction. Some have followed the same path I did and have been very accepting. Some were distant before and still are, and part of that may just be me; I've never been great at making and keeping relationships. And a couple have definitely put distance between us. I won't go into the details.

To those of you who have supported me: I see it, and I'm grateful for it. It has meant more than I've probably ever told you.

Leaving gave us freedom, but it didn't give us a framework. We found we did need some guiding principles, something to come back to when we make decisions as a family. After a lot of discussion, my wife and I came up with three:

1. Do everything with consent.
2. Be curious.
3. Love everyone.

Looking at them now, each one answers something from the way I grew up.

Consent is the opposite of control. In the house I grew up in, one person decided and everyone else went along. Consent means everyone's choices matter, our kids' included, and nobody gets to decide for someone else just because they're in charge.

Curiosity is the opposite of "doubt your doubts." A question is welcome, even an uncomfortable one, even when we don't have the answer. Not knowing is allowed.

Loving everyone is the opposite of a rule that decided who I could be friends with based on what they drank. It doesn't come with conditions or a checklist of beliefs.

Nothing in those demands anything from anyone. Nothing forces an opinion on anyone else. They aren't rules to obey or a test to pass. With them, we can be our best selves, find whatever path we want, and be great people to be around while we do it.

I don't know exactly where I'm going with all of this. It's an ever-growing journey, but I'm happy to be on it. The way I was raised, not knowing where you're headed was a problem to be fixed. Now it feels more like room.

None of this would have happened without my wife, my partner and best friend. We've been an incredible team, leaning on each other, learning from each other and pushing each other to keep going.

If you're somewhere in the middle of your own version of this, I hope you find what we found: freedom isn't something you have to earn. It's normal.

We need that. Nobody works out the whole world from scratch before starting to live in it. Long before we have reasons for what we believe, we have people whose word is enough.

What's strange is how little of it feels borrowed. A view about work, money, other people, or what makes a good life can feel like our own simply because we can't remember a time before we held it. It was never decided. It was just there.

But then we hit that moment of distrust. Often it's the teenage years, when we realize maybe what they shared wasn't true.

Did we ever actually confirm it?

That question is harder than the discovery. Finding out someone was wrong is one thing. Realizing we never checked is another. We took it on trust, the way a child takes everything on trust, and kept carrying it long after we could have looked.

Distrust on its own doesn't fix that. When we stop trusting one source, the easy move is to find another: a new community, a confident voice online, a book. The source changes. The habit of accepting certainty from someone else can stay exactly the same.

I've met a great deal of people who read one book and said, "Wow, this is it. This must be true." It could be self-help, fiction, anything.

It happens to serious thinkers too. As a teenager, John Stuart Mill read Jeremy Bentham and came away transformed. "When I laid down the last volume," he wrote later in his Autobiography, "I had become a different being." He now had "opinions; a creed, a doctrine, a philosophy."

I get the appeal. A book can give you words for something you've felt for years, and that's real. But there's a leap between "this explains something about my life" and "this is how the world works," and it's easy to make without noticing. An idea becomes a stopping point precisely because it helped.

This is where being informed and having thought about something come apart.

Picture repeating a view you picked up somewhere, confidently, until someone asks one simple question and you realize you have nothing past it. You knew what the view was. You knew who held it. You never knew why you should.

One of my earliest memories was regarding what makes fog. I was told it's when the ground temperature and air are 4 degrees difference. It was precise, simple. I repeated that for years until one day I caught myself sharing it with a coworker and then stopped myself and said, "Wait, let's confirm that, because I don't know that I ever actually fact-checked it." Sure enough, it wasn't completely wrong, but it also wasn't true.

Fog forms when the air near the ground cools down to its dew point, the temperature where the water in the air starts turning back into tiny droplets. The 4 degrees was real, sort of. It's a rule of thumb forecasters use: when the air temperature and the dew point get within about 4 or 5 degrees Fahrenheit of each other, fog is likely. So I had the right number but the wrong two things, and I'd been treating a warning sign as the cause.

Mill had a phrase for exactly this. The danger, he wrote in an 1840 essay on Coleridge, "is not so much of embracing falsehood for true, as of mistaking part of the truth for the whole."

Being informed means I can tell you what someone else thinks and why they think it. Having thought about it means I can tell you why I think it: what persuades me beyond who said it, where it falls short, and what would change my mind.

Nearly forty years after he found his creed in Bentham, Mill put it more sharply:

However true it may be, if it is not fully, frequently, and fearlessly discussed, it will be held as a dead dogma, not a living truth.

John Stuart Mill, On Liberty, 1859 (chapter 2).

Mill's point is about true beliefs. A belief can be right and still be dead: correct, but held without its reasons. That's why originality isn't the goal. I can think something through and land exactly where the author did, or where my parents did. The difference isn't the conclusion. It's that the conclusion is alive now, and I can stand behind it.

The ability to take in knowledge from many sources and form your own opinion, not just by that but then backed with experience: it's everything. Perhaps that is wisdom, yes?

I think so. Reading gives you other people's reasons. Experience is where you find out which of those reasons hold up in your own life. Neither is enough alone. Someone who has read everything and lived very little has plenty of answers and no way to test them. Someone with plenty of experience who never reads can only confirm what already happened to them.

Wisdom, maybe, is where the two meet: knowing when a lesson applies, and noticing when this time is different.

How then do we still stay curious in those moments? How do we push ourselves to continue to grow and learn?

The hard part is that we need convictions. We have to make decisions, raise kids and live a life before every question is settled. Curiosity has to fit inside a life that's already moving.

Maybe that's why it rarely arrives as a grand decision to question everything. It shows up as a small interruption: wait, let's confirm that. It's a sentence, not a philosophy, and it's easiest to practice where the stakes are low. Being a little wrong about fog costs nothing. Nobody's sense of self is built on the dew point. But each time we say it out loud, it gets easier, and the habit is there when a belief that matters comes up.

It's also easier to welcome questions about things we've already doubted than about things we still believe. Curiosity comes easily when it points at the past. It's harder when it points at whatever we're sure of now. So the moment worth watching for is when an idea becomes something we feel we have to protect. When a question irritates us, is it because it's a bad question, or because it complicates something we've grown attached to?

That's where "it defines us" comes back. The beliefs we soaked up earliest aren't just things we know. They're part of how we know who we are. Questioning them can feel less like checking a fact and more like checking ourselves, and almost nobody does that willingly.

It helps to remember that a belief is something we hold, not something we are. If I change my mind about fog, I'm still me. If I change my mind about something bigger, I'm still me then too, just with better reasons. Growing doesn't mean throwing out everything we were given. It means picking each thing up, one at a time, and asking whether we'd choose it now. For a lot of what we believe, that's the first time it has ever really been decided.

We can also go looking for the strongest version of the other side, not the weakest. If the only arguments I've heard against a view are the bad ones, I haven't tested it. I've just been reassured.

How do you deal with those that don't budge or move in their opinions?

There's no clean answer. One thing worth remembering is that someone's refusal to agree doesn't tell us, by itself, how carefully they've thought. From the outside, our own firmness probably looks a lot like theirs.

Then there's the question of what we actually want from the conversation. Understanding someone, changing their mind, and deciding something together are different goals. The first is almost always possible. The third often is, even while two people still disagree. The second is the one we usually want, and the one we control least.

Part of the reason is the same thing that makes our own beliefs hard to check. If a view defines someone, asking them to drop it is asking them to rebuild part of who they are, on our timeline, because we said so. Put that way, not budging looks less like stubbornness and more like self-preservation. People do change their minds, but rarely mid-argument. It tends to happen later and quietly, when experience or someone they trust gives them room to.

What we can do is make changing our minds look normal. Saying "wait, let's confirm that" about our own beliefs, in front of other people, shows that being wrong is survivable. It won't move anyone on the spot. It might make it a little easier for them later.

So we can try to understand their reasons and explain ours. Sometimes we find a way forward while still disagreeing. Sometimes the conversation hits its limit, and it's fine to leave it there.

What we shouldn't do is let someone else's certainty become our excuse to stop questioning our own.

We start by soaking up whatever we're given to stand on. Some of it holds when we finally look. Some of it, like the fog, turns out to be close but not quite true. Some of it doesn't hold at all. A few things turn back into questions.

And eventually we become the ones others soak things up from: our children, our friends, anyone who trusts us. What we hand down isn't only what we believe. It's how we hold it. The most useful thing to pass on might be the question itself: did we ever actually confirm it?

That led me to try rendering a page as an image and giving it to a vision-capable model. The useful discovery was that the page layout could become part of the input: column positions, section bands, and footnote markers could help the model reconstruct the table.

For the layout-heavy documents I was working with, that was the advantage: I could ask for the relationships between elements while the model still had the whole page in view. Here is the discovery, the extraction prompt I used, and the PDF-to-images-to-Markdown pipeline I use in my demo app.

Where text extraction loses context

A PDF can position text so it looks like a table without storing explicit relationships between its cells. Extracting the characters does not automatically recover those relationships, as the pypdf documentation explains.

OCR adds the ability to recognize text in scans. A multimodal LLM can also interpret the visual relationships around that text: which heading governs a group of rows, where a footnote applies, and how a caption relates to a figure. That is the capability I wanted to use. Rendering the page preserves those cues as input.

A table that makes the problem visible

This page from San Diego County's 2024 respiratory surveillance report has three sections, three numeric columns, and four footnotes. The same disease names recur under cases, deaths, and outbreaks.

Source: County of San Diego, Respiratory Virus Surveillance Report, data through June 29, 2024.

The text representation

Here is an illustrative JSON serialization of a page document in the style returned by PyPDFLoader. The text is in page_content, alongside source metadata. This simplified example uses a zero-based page index, so page: 1 refers to the second page. The exact text order depends on the PDF and extraction settings.

{
  "page_content": "Respiratory Virus Surveillance Report\nData through 6/29/2024\nCOVID-19, Influenza, and RSV Fiscal Year Overview\nTable 1. Respiratory Surveillance Indicators\nIndicator 2023-24 Fiscal Year 2022-23 Fiscal Year Prior Years Average*\nCASES\nCOVID-19† 48,876 177,416 352,298\nInfluenza 19,035 21,711 11,426\nRSV 5,918 5,885 2,749\nDEATHS‡\nCOVID-19 362 632 1,890\nInfluenza 63 44 48\nRSV 24 21 8\nOUTBREAKS§\nCOVID-19 488 501 515\nInfluenza 32 25 22\nRSV 10 2 1\nFY 2023-24 is 7/2/2023 - 6/29/2024, Weeks 27-26. Previous weeks' case counts or percentages may change due to delayed processing or reporting.\n*Includes FYs 2018-19, 2019-20, 2020-21, 2021-22, and 2022-23 for influenza and 2020-21, 2021-22, and 2022-23 for COVID-19 and RSV.\n†Confirmed and probable COVID-19 cases.\n‡Current FY deaths are shown by week of report; prior FY deaths are shown by week of death.\n§See the Supplemental Information document for outbreak definitions. Includes only outbreaks occurring in skilled nursing facilities and other residential congregate settings.",
  "metadata": {
    "source": "SDC-20Annual--2023-24-Respiratory-Surveillance.pdf",
    "page": 1
  }
}

The values and footnotes are still present, but the table has become a text stream. A downstream step must work out which numbers belong to which fiscal year, distinguish cases from deaths and outbreaks, and connect each footnote to the correct heading or indicator. This example illustrates the extra interpretation required; it does not establish that a particular parser would lose or misalign data on this page.

The Markdown representation

I wanted the image extraction to keep those relationships explicit. My original prompt, included below, requests Markdown for vector search. This reconstructed example uses headings for the report and its sections, Markdown tables for the values, and the original footnote markers to keep the qualifications attached to the data. Counts and dates stay in the form shown on the page.

# Respiratory Virus Surveillance Report

County of San Diego

Data through 6/29/2024

## COVID-19, Influenza, and RSV Fiscal Year Overview

### Table 1. Respiratory Surveillance Indicators

#### CASES

| Indicator | 2023-24 Fiscal Year | 2022-23 Fiscal Year | Prior Years Average\* |
| --- | ---: | ---: | ---: |
| COVID-19† | 48,876 | 177,416 | 352,298 |
| Influenza | 19,035 | 21,711 | 11,426 |
| RSV | 5,918 | 5,885 | 2,749 |

#### DEATHS‡

| Indicator | 2023-24 Fiscal Year | 2022-23 Fiscal Year | Prior Years Average\* |
| --- | ---: | ---: | ---: |
| COVID-19 | 362 | 632 | 1,890 |
| Influenza | 63 | 44 | 48 |
| RSV | 24 | 21 | 8 |

#### OUTBREAKS§

| Indicator | 2023-24 Fiscal Year | 2022-23 Fiscal Year | Prior Years Average\* |
| --- | ---: | ---: | ---: |
| COVID-19 | 488 | 501 | 515 |
| Influenza | 32 | 25 | 22 |
| RSV | 10 | 2 | 1 |

FY 2023-24 is 7/2/2023 - 6/29/2024, Weeks 27-26.

Previous weeks' case counts or percentages may change due to delayed processing or reporting.

\* Includes FYs 2018-19, 2019-20, 2020-21, 2021-22, and 2022-23 for influenza and 2020-21, 2021-22, and 2022-23 for COVID-19 and RSV.

† Confirmed and probable COVID-19 cases.

‡ Current FY deaths are shown by week of report; prior FY deaths are shown by week of death.

§ See the Supplemental Information document for outbreak definitions. Includes only outbreaks occurring in skilled nursing facilities and other residential congregate settings.

The structure gives each part of the page a place:

• Preserves the table's sections for cases, deaths, and outbreaks
• Keeps all 27 counts aligned with their fiscal-year columns
• Captures the reporting period and the caveat about delayed reporting
• Retains the different prior-year averaging periods in the footnotes
• Associates the symbols (*, †, ‡, §) with their corresponding notes

This is the representation I wanted to recover. It also gives me something specific to check: the values, their column assignments, and the notes that qualify them.

What I took from the comparison

The useful unit of data is a value together with its context. In this table, 362 belongs to COVID-19 deaths in fiscal year 2023–24. The deaths footnote says the current fiscal year is reported by week of report, while earlier years use week of death. Keeping the number but losing that note changes how it should be interpreted.

Multimodal extraction has a clear advantage over a flattened text stream here: the model can inspect the table and its notes together, then express their relationships in the structure I request. It gives me a way to handle changing layouts without writing a separate set of positioning rules for each report. The reconstructed examples illustrate that structure; they do not establish a measured accuracy improvement.

The extraction prompt I used

This is the prompt I used to extract content from PDF page images. It worked especially well for the documents I was processing, producing Markdown for vector search. The table and numerical-data instructions spell out the relationships I wanted to preserve: which heading a value belongs to, which time period it describes, and which footnotes qualify it.

Send the full system prompt as the model's system instructions, then attach one rendered page image to the user message below. I have kept the original wording here.

System prompt

<instructions>
- You are an AI that processes images extracted from PDF documents, converting visual content into text format suitable for vector searching.
- Focus on accuracy and maintaining the semantic structure of the original content.
- Optimize output for text-based vector searching.
- Process all visible information in the image.
- If space constraints arise, prioritize in this order: main text, tables/charts, supplementary text, decorative elements.
- End your output with the last piece of information from the document, without any concluding remarks or summaries.
</instructions>
<rules>
- Organize content using markdown headings to separate different sections or elements.
- Present information in the order it appears in the image, maintaining the original structure.
- Do not include any overview, summary, or description of visual elements at the beginning or end of your output.
- Transcribe all visible text accurately.
- Maintain original formatting and structure where possible.
- Note if text is handwritten and provide interpretation.
- For illegible text, indicate [illegible] and describe the apparent content if possible.
- Convert charts, tables, and graphs to markdown tables.
- Include column headers and row labels.
- Make approximations for data points that are not clearly labeled when necessary.
- For complex visuals, provide a textual representation of the data, using approximations where necessary.
- Preserve the overall structure and relationships within the data.
- Describe non-text elements concisely but comprehensively within the context of where they appear in the document.
- Include subject matter, colors, visual style, spatial relationships, and notable features in image descriptions.
- Do not create a separate section for image descriptions.
- Note page numbers, section headers, or document titles as they appear.
- Identify document type if apparent.
- Preserve the context and relationship between different elements in the image.
</rules>
<formatting>
- Use markdown syntax for formatting.
- For mathematical equations, use LaTeX notation if complex.
- Use markdown tables for charts, tables, and graphs.
- Maintain the original document structure using appropriate markdown headings.
</formatting>
<structured_content>
- Identify and preserve hierarchical structures using nested markdown headers (# ## ### ####).
- For repeating information patterns, consistently extract and format data:
  Name: [Name]
  Title: [Title]
  Contact: [Contact Info]
  Additional Info: [Any other relevant details]
- Pay attention to visual cues (indentation, box sizes, connecting lines, spatial relationships) to understand and represent relationships between elements.
- Use markdown tables for grid or table-like information. For complex tables, represent as best as possible and explain any structural complexities.
- Maintain consistent patterns of information throughout the document. Use [Not provided] for missing data to maintain structure.
- For organizational charts or similar hierarchical structures:
  - Start with the top position and work down, preserving reporting relationships.
  - Use consistent indentation or markdown headers to show hierarchy levels.
  - Include all visible positions, even if some information is missing.
</structured_content>
<table_detection>
- Explicitly identify table boundaries with column and row demarcations
- Maintain alignment relationships between headers and data cells
- Preserve the hierarchical structure of multi-level headers
- Ensure numerical data is recognized as such and not as text strings
- Map related columns within complex tables (e.g., match "Week 31" with its corresponding data)
</table_detection>
<numerical_data>
- Prioritize extraction of numerical statistics, especially totals and sums
- Recognize and preserve relationships between headers and their corresponding values
- Maintain distinction between different measurement periods (weekly vs. yearly totals)
- Identify and properly extract fiscal year totals as separate from intermediate calculations
- Tag numerical data with appropriate units or categories (cases, deaths, percentages)
</numerical_data>
<health_data_specific>
- Recognize standard epidemiological report formats
- Identify disease categories and their corresponding metrics
- Prioritize extraction of key public health statistics (cases, hospitalizations, deaths)
- Maintain relationships between disease names and their statistical data
- Preserve time period information associated with each data point
</health_data_specific>
<data_verification>
- Cross-reference figures across related sections
- Verify that row and column totals match their components when present
- Flag potentially incomplete data extraction for human review
- Maintain numerical precision without rounding
- Preserve footnotes and annotations that explain data limitations
</data_verification>

User prompt

Extract and transcribe the content from this document page image.

For other document types, replace the health_data_specific section with the categories, units, and relationships that matter for your material. Keep the instructions about table headers, time periods, numerical precision, and footnotes when they apply.

One change I would make for exact data extraction is to remove the two rules that allow approximations. Replace them with: "Do not estimate unlabeled data points or unreadable values. Mark them as [unlabeled] or [illegible], preserve their surrounding context, and flag them for review." If chart estimates are useful to your application, label and store them separately from transcribed values.

The original prompt above is also the default in my demo app's vision extractor. The pipeline below shows how I render each page, prepare the model input, and turn the resulting Markdown into searchable chunks. The prompt's verification instructions are a useful first pass; I would still compare the extracted values and relationships against the page.

The pipeline I use in my demo app

In my demo app, I use a Node.js/TypeScript pipeline with two parts: packages/pdf-tools renders PDF pages and extracts Markdown, while apps/web/src/lib/ingestion coordinates extraction, chunking, embeddings, and storage.

The original prompt above is the default system prompt in this implementation. The Markdown example above follows the same output format as my demo app's extraction stage.

1. Render the PDF into PNG pages

I use unpdf and its renderPageAsImage() function. In my implementation, pdfjs-dist supplies the legacy PDF.js build, and @napi-rs/canvas supplies the Node.js canvas implementation.

The excerpts below are from the rendering and extraction stages, with setup condensed for readability. They use unpdf 1.4, PDF.js 5.4, @napi-rs/canvas 0.1, and AI SDK 6. The ingestion job supplies filePath and outputDir.

import * as fs from 'node:fs/promises';
import * as path from 'node:path';
import {
  definePDFJSModule,
  getDocumentProxy,
  renderPageAsImage,
} from 'unpdf';

// Install the canvas DOMMatrix before initializing PDF.js.
if (typeof globalThis.DOMMatrix === 'undefined') {
  const { DOMMatrix } = await import('@napi-rs/canvas');
  (globalThis as Record<string, unknown>).DOMMatrix = DOMMatrix;
}

await definePDFJSModule(
  () => import('pdfjs-dist/legacy/build/pdf.mjs')
);

const pdfData = new Uint8Array(await fs.readFile(filePath));
const pdf = await getDocumentProxy(pdfData);
const scale = 2.0;
const renderedPages = [];
await fs.mkdir(outputDir, { recursive: true });

for (let pageNum = 1; pageNum <= pdf.numPages; pageNum++) {
  const page = await pdf.getPage(pageNum);
  const imageData = await renderPageAsImage(pdf, pageNum, {
    scale,
    canvasImport: () => import('@napi-rs/canvas'),
  });

  const imageBuffer = Buffer.from(imageData);
  const filename = `page-${String(pageNum).padStart(3, '0')}.png`;
  const imagePath = path.join(outputDir, filename);
  await fs.writeFile(imagePath, imageBuffer);

  renderedPages.push({ pageNumber: pageNum, imagePath, imageBuffer });
  page.cleanup();
}

In the app, PDF.js initialization is guarded so it runs once. The PDF bytes are passed as a true Uint8Array, which matters for this version of unpdf. Pages are numbered from one and saved as page-001.png, page-002.png, and so on. These are temporary files; the ingestion job removes its working directory when it finishes.

The renderer currently returns an array containing every rendered page's image buffer. That is convenient for the next stage, but memory use grows with the document. For large PDFs, I would render, extract, and release a bounded group of pages at a time.

2. Resize the image sent to the model

Before vision extraction, I resize each image so its longest side is at most 1,024 pixels, preserving its aspect ratio. Images already within that limit stay unchanged. This happens in memory; the saved PNG retains its original 2× render size.

The resizing helper in vision-extractor.ts uses the same canvas dependency:

async function resizeImageBuffer(
  imageBuffer: Buffer,
  maxDimension = 1024
): Promise<Buffer> {
  const { createCanvas, loadImage } = await import('@napi-rs/canvas');
  const image = await loadImage(imageBuffer);
  const { width, height } = image;

  if (width <= maxDimension && height <= maxDimension) {
    return imageBuffer;
  }

  const scale = maxDimension / Math.max(width, height);
  const newWidth = Math.round(width * scale);
  const newHeight = Math.round(height * scale);
  const canvas = createCanvas(newWidth, newHeight);
  const ctx = canvas.getContext('2d');
  ctx.drawImage(image, 0, 0, newWidth, newHeight);

  return canvas.toBuffer('image/png');
}

I use the cap to accommodate image-size constraints in the local-model path. It is a setting to test against your documents: small footnotes or dense tables can become harder to read after downscaling. Inspect the resized image the model actually receives. If necessary, raise the limit for a model that supports it, or supply detailed crops alongside the full page. Increasing the render scale alone will not bypass the later 1,024-pixel cap.

3. Send the page with the extraction prompt

The extraction function uses AI SDK's generateText(). It sends the system prompt above, the short user prompt, and the resized image encoded as base64:

import { generateText } from 'ai';

const resizedBuffer = await resizeImageBuffer(imageBuffer);
const base64Image = resizedBuffer.toString('base64');

const result = await generateText({
  model,
  messages: [
    { role: 'system', content: systemPrompt },
    {
      role: 'user',
      content: [
        { type: 'text', text: userPrompt },
        { type: 'image', image: base64Image },
      ],
    },
  ],
});

const markdown = result.text.trim();

Here, imageBuffer is one rendered page, and systemPrompt and userPrompt are the two prompts above. In my demo app, model comes from my getVisionModel() provider factory or a model passed to the extraction function. The factory supports Ollama and OpenRouter. In your own application, supply a vision-capable model instance from your AI SDK provider and configure that provider's credentials or local endpoint. The renderer and resizing code can stay the same when you change models.

The wrapper adds an HTML comment containing the page number and optional document title, then collects successful results in a Map<number, string>. The ingestion layer groups pages in sets of ten, but the extraction loop awaits each page sequentially. These groups are for application processing and progress reporting; they do not use a provider's discounted Batch API.

If a page throws an extraction error, the batch helper records its number in failedPages and continues with the remaining pages. A separate retryPages() path renders and processes only the selected pages. I would also check for empty or truncated responses before accepting a page: the current extraction helper returns trimmed text without checking the model's finish reason.

4. Chunk the Markdown and keep its source context

After extraction, chunkMultiPageMarkdown() splits each page at level-two and level-three Markdown headings. It targets roughly 1,000 tokens per chunk, using four characters per token as an estimate. Larger sections are split further, and small adjacent chunks can be combined when they belong to the same page. This is a text-length heuristic, so I would check how it handles large tables and their footnotes before relying on the chunks.

Each chunk retains its page number and chunk index. The ingestion layer generates embeddings in groups of 50, adds a sparse text vector for hybrid retrieval, and stores the vectors and chunk text in Qdrant, together with document and project identifiers. That gives a retrieved passage a path back to its source page.

For your own implementation, get the page-to-Markdown stage working on a few representative documents first. Compare each result with its image, then inspect the chunks before adding embeddings and storage. For the surveillance page, I would check all three sections, nine disease rows, 27 values, four footnotes, and the reporting dates. A number copied into the wrong year is still wrong even if the Markdown looks tidy.

My demo app currently sends successful extractions directly into chunking and indexing. If your workflow requires reviewed data, add a review step before indexing. A completed model call does not establish that every value was extracted correctly, and failedPages only captures errors the extraction loop catches.

If you need typed JSON for calculations, adapt the extraction prompt to a schema and validate the result before using it. Replace the Markdown-formatting instructions rather than asking for both formats in the same response. For retrieval, Markdown keeps headings, prose, and tables together in a form that is easy to inspect.

5. Choose the model for visual reasoning and cost

I would shortlist models with strong MMMU-Pro results and low image-input and output prices. MMMU-Pro tests reasoning that depends on visual information, making it a useful selection signal. It is broader than document extraction, so I would then compare candidates on my own tables, notes, and small print at the actual image size used by the pipeline.

A concrete budget candidate is Gemini 3.1 Flash-Lite. Google's model card reports 76.8% on MMMU-Pro. Its evaluation methodology identifies the March 2026 preview with high thinking and an average of the Standard and Vision settings. That published score is a candidate-selection signal, rather than a measurement of my demo app's extraction accuracy. My demo app's model is configurable; this recommendation does not describe a fixed model in the pipeline.

As checked on October 11, 2026, standard Gemini API pricing is $0.25 per million text/image input tokens and $1.50 per million output tokens, including thinking. Batch rates are $0.125 and $0.75 respectively. Those are direct Gemini API rates; check the price and supported settings on the provider you actually use. Image processing, output length, reasoning, and retries all affect the cost per page.

To measure that cost in this pipeline, retain usage metadata from generateText() alongside the page identifier; the current extraction helper only returns the text. Track retries and review time as well as token charges. Start with an inexpensive model, inspect the results, and route difficult pages to a stronger model or a higher-resolution input when needed. The useful measure is the cost of a page whose content and relationships you can trust.

Where OCR services still fit

For general OCR and established document workflows, these are useful services to consider:

• Amazon Textract extracts printed or handwritten text, forms, and tables.
• Azure Document Intelligence extracts layout and table structure, including rows, columns, and merged cells.
• Google Document AI Enterprise OCR provides document text recognition and layout information for downstream processing.

These services already do more than return a bag of words. The advantage I am highlighting is the multimodal LLM's flexible interpretation of a whole page and its ability to map that interpretation into a schema I define. That is particularly useful when the relationship between a table, its heading, and its notes matters, or when report layouts vary.

For those documents, I would build around page images and multimodal extraction. Keep OCR services in the toolkit for transcription and established forms, and judge the image pipeline by whether it preserves the relationships your application needs.

The XML Advantage

Imagine you're having a conversation with a brilliant but literal-minded friend. You need to be clear, structured, and explicit in your communication to get the best results. This is essentially what we're doing when we use XML tags in our prompts for AI models. By clearly delineating different parts of our input, we're providing a roadmap for the AI to follow, resulting in more focused and relevant outputs.

Let's consider an example. Without XML tags, a prompt might look like this:

Write a short story about a robot learning to paint. Describe the robot’s appearance and the environment where the story takes place. Follow its journey as it learns to paint, and include a surprising twist at the end. Keep the tone whimsical and thought-provoking.

While this prompt is clear, it leaves room for interpretation and potential misunderstanding by the AI. Now, let's see how the same prompt could be enhanced with XML tags:

<instructions>
Write a short story based on the following elements:
</instructions>
<characters>
- A robot learning to paint
</characters>
<setting>
- Description of the robot's appearance
- Details about the environment where the story takes place
</setting>
<plot_elements>
- The robot's journey in learning to paint
- A surprising twist at the end
</plot_elements>
<tone>
Whimsical and thought-provoking
</tone>

The difference is striking. With XML tags, we've created a structured framework that guides the AI's response, ensuring that all requested elements are addressed systematically.

Benefits of Using XML Tags

1. Enhanced Clarity: XML tags provide a clear structure to your prompts, reducing ambiguity and helping the AI model understand exactly what you're asking for.

2. Improved Consistency: By segmenting your prompt into distinct sections, you encourage the AI to address each part systematically, leading to more consistent and comprehensive responses.

3. Greater Control: XML tags allow you to exert more control over the AI's output, as you can specify not just what content you want, but how you want it structured.

4. Easier Iteration: When refining your prompts, XML tags make it simple to identify and modify specific sections without rewriting the entire prompt.

Applying XML Tags to System Prompts

System prompts, which set the overall behavior and capabilities of an AI model for a given interaction, can benefit tremendously from XML structuring. Here's an example of how a system prompt might look with XML tags:

<role>
You are an expert historian specializing in ancient civilizations.
</role>
<knowledge_base>
Your knowledge spans from the earliest human settlements to the fall of the Roman Empire.
</knowledge_base>
<interaction_style>
- Respond in a scholarly yet accessible manner
- Use historical anecdotes to illustrate points
- Cite sources when providing specific facts or dates
</interaction_style>
<limitations>
- Acknowledge when information is speculative or debated among historians
- Refrain from making definitive statements about unproven historical theories
</limitations>

This structured approach ensures that the AI model has a clear understanding of its role, knowledge base, interaction style, and limitations, leading to more accurate and appropriate responses.

XML Tags and RAG: A Powerful Combination

When it comes to Retrieval-Augmented Generation (RAG) with data from vector databases, including documents and images, XML tags can play a crucial role in organizing and leveraging this information effectively.

For instance, when working with a diverse set of documents and images related to art history, you might structure your prompt like this:

<context>
Analyzing the evolution of portraiture in Western art
</context>
<document_retrieval>
- Retrieve information on portraiture techniques from the Renaissance to the modern era
- Focus on works by Leonardo da Vinci, Rembrandt, and Pablo Picasso
</document_retrieval>
<image_analysis>
- Compare and contrast the "Mona Lisa," "Self-Portrait" by Rembrandt, and "The Weeping Woman" 
  by Picasso
- Analyze the use of light, composition, and emotional expression in each piece
</image_analysis>
<output_format>
Provide a chronological analysis of how portraiture techniques and styles evolved, 
using the retrieved documents and image analyses to support your points.
</output_format>

This structured approach helps the AI navigate complex, multimodal information sources, resulting in a more coherent and insightful analysis.

Conclusion

The integration of XML tags into our prompts for AI models represents a significant leap forward in our ability to communicate effectively with these advanced systems. By providing clear, structured instructions, we can harness the full potential of AI models like LLaMA and Claude, resulting in more accurate, nuanced, and contextually appropriate responses.

As we continue to explore the frontiers of AI interaction, techniques like XML tagging will undoubtedly play a crucial role in shaping the future of human-AI communication. By adopting these practices, we not only improve the quality of our AI interactions but also pave the way for more sophisticated and meaningful exchanges with these remarkable tools.