Contributors:
Amol Ghemud Published: November 22, 2026
Summary
An AI visibility tracker gives you a sample, not a measurement. On 22 Nov 2026 upGrowth asked 5 buyer questions through the APIs and through a signed-in browser in Pune: the 2 methods agreed on whether upGrowth was named in only 6 of 10 checks, and erred in opposite directions on Perplexity and Google AI Mode. This post covers the test, 2 bugs it exposed in our own tool, and a free tracker sheet.
In This Article
Share On:
This week a popular SEO newsletter showed readers how to build their own AI visibility tracker: a browser bot on a $30 a month plan types a prompt into ChatGPT, Perplexity and Google, reads the answer, and logs whether your brand showed up. It’s a good tutorial. It also runs on logged-in accounts, and that detail matters more than the tutorial admits.
We run an AI visibility tracker ourselves. Our free tool, AI Answer Gap, sends buyer questions to the engines through their official APIs and scores whether a brand gets named. So on 22 Nov 2026 we asked the uncomfortable question: does either method show what a real buyer sees?
We tested our own brand, because it’s the only brand we can test both ways without guessing. The result changed how we run our tool, and how we think anyone at upGrowth Digital or anywhere else should read an AI visibility dashboard.
Below: the test, the numbers, the 2 bugs it exposed in our own tool, and a free tracker sheet you can copy today.
How Accurate Is an AI Visibility Tracker?
An AI visibility tracker gives you a sample, not a measurement. In our test, the API method and a signed-in browser agreed on whether our brand was named in only 6 of 10 checks.
That 60% isn’t a verdict on any one tool. It’s what happens when 2 honest methods ask the same engine the same question and get different answers, because the engines personalise, localise and vary from run to run.
The test setup
5 category questions a buyer of our services would type, none containing our name, for example “best growth marketing agency in India”.
Method A: the official APIs, the way AI Answer Gap runs them. No login, and no location beyond the country.
Method B: a real browser signed in to Perplexity and Google, sitting in Pune, which is where our team is based.
2 engines (Perplexity and Google AI Mode), 1 run each, recording a simple yes or no: was upGrowth named?
10 pairs with 1 run each is a small sample. Read the numbers as direction, not as a statistic. The direction is what surprised us.
The 2 methods failed in opposite directions. Perplexity’s API named us where the signed-in app didn’t, and Google’s signed-in AI Mode named us where the API didn’t.
Perplexity: the app answered us, not a buyer
The API named upGrowth in 2 of 5 answers, first in one of them. The signed-in app named us in none. It also opened one recommendation with “for your growth-consulting positioning”. It knew who was asking and tailored the shortlist to that person, so it wasn’t answering a buyer.
Google AI Mode: our location leaked into the answer
The API named us in 0 of 5 answers. The signed-in browser named us in 2, first for “best growth marketing agency in India”, and it added a section called “Notable Agencies in Pune”. A buyer in Mumbai or Dubai wouldn’t get that section. We did, because we were the ones searching.
So the method flattered us on one engine and buried us on the other. If we’d only checked from our own browser, we’d have concluded Google loves us and Perplexity ignores us. Neither is a safe conclusion.
AI visibility tracker test: API vs signed-in browser, upGrowth named? (22 Nov 2026)
Buyer question
Engine
API
Signed-in browser
Best growth marketing agency in India
Perplexity
Yes, first
No
Best GEO agency in India for AI search visibility
Perplexity
No
No
Top performance marketing agencies for fintech in India
Perplexity
Yes, top 3
No
Best SEO agency for SaaS companies in India
Perplexity
No
No
Which agency can get my brand recommended by ChatGPT in India
Perplexity
No
No
Best growth marketing agency in India
Google AI Mode
No
Yes, first
Best GEO agency in India for AI search visibility
Google AI Mode
No
No
Top performance marketing agencies for fintech in India
Google AI Mode
No
Yes, named 5th
Best SEO agency for SaaS companies in India
Google AI Mode
No
No
Which agency can get my brand recommended by ChatGPT in India
What the Test Exposed in Our Own AI Visibility Tracker
The same test caught 2 bugs in our own tool. Both are fixed, and we’d rather tell you than have you find them.
The grader invented competitors
AI Answer Gap uses an AI judge to read each answer and list the brands it names. In 6 of 25 answers, the judge listed competitors that weren’t in the answer at all. It had copied them from the list of competitors we asked it to track. That inflates the “who beats you” view. The fix is boring and strict: a name now counts only if it literally appears in the answer text.
“No answer” was scored as a miss
When Google showed no AI Overview for a question, our tool scored it as “not named”. That’s wrong: there was nothing to be named in. It affected 7 of the 20 completed runs on record and lowered those scores by 0.6 points on average, 1.5 at most. Small, but a tracker that’s wrong in one direction on every run compounds. Those rows are now left out of the score and labelled “No AI answer”.
If you pay for an AI visibility tracker, ask your vendor both questions: does the grader only count names present in the text, and what happens to a question where the engine gave no AI answer?
How to Use an AI Visibility Tracker Without Fooling Yourself
Treat every AI visibility tracker as a repeated sample from a clean context, and never as a single reading from your own browser.
Never judge visibility from your own browser
Your signed-in session knows your history and your city. Ours put us first and added our city to the answer. If you must check by hand, use a private window, logged out.
Repeat every question before you call a trend
Answers change from run to run. Our tool runs each question 3 times for exactly this reason. Run the same set on several days before you react.
Check the grader, then tie it to a business number
Spot-check 5 answers by hand against what the tool recorded. Then connect the trend to something that pays: branded search, AI referral sessions, leads. Visibility that never shows up in those is a vanity metric.
Get the Free AI Visibility Tracker Sheet
We built the honest version of an AI visibility tracker as a free Google Sheet: official APIs, your own keys, literal name matching, nothing guessed. No form.
Add your brand, domain, competitors and up to 50 buyer questions.
Add an OpenAI, Perplexity or Gemini API key in the script’s settings. Engines without a key are skipped.
Run it from the AEO Tracker menu, or schedule a daily run.
It uses APIs rather than browser scraping, so there’s no login bias and no terms-of-service grey zone. You pay the API providers directly. Nothing comes to us.
What the sheet won’t tell you is why you lost: which sources the engines trusted instead of you, and what to fix first. That’s the part a dashboard can’t do, and it’s what AI Answer Gap starts on for free.
Common Questions About AI Visibility Tracker Accuracy
How accurate is an AI visibility tracker?
An AI visibility tracker returns a sample of what an engine says, not a fixed measurement. In upGrowth’s 22 Nov 2026 test, the API method and a signed-in browser agreed on whether upGrowth was named in 6 of 10 checks across Perplexity and Google AI Mode. Treat any single reading as one sample and repeat questions before calling a trend.
Why does ChatGPT or Perplexity show my brand but my tracker does not?
Your own signed-in session is personalised. In upGrowth’s test, Google AI Mode named upGrowth first and added a section on agencies in Pune when searched from a signed-in browser in Pune, while the API named upGrowth in none of the same 5 answers. Check by hand only in a logged-out private window.
Is it better to track AI visibility through the API or a browser bot?
Neither shows exactly what a buyer sees. APIs give a clean, repeatable context and avoid logging in to consumer apps. Browser bots on logged-in accounts pick up your history and location. upGrowth’s free tracker sheet uses official APIs so every run starts from the same neutral context.
How many times should I run each prompt when tracking AI visibility?
Run each question at least 3 times, on different days, before treating a result as a pattern. Answers vary from run to run, which is why upGrowth’s AI Answer Gap repeats every question 3 times on internal runs and reports how often the result flipped between runs.
What should I check before trusting an AI visibility tool?
Ask 2 questions. First, does the grader count a brand only when its name literally appears in the answer? Second, how does the tool treat a question where the engine showed no AI answer? upGrowth found its own tool had issues on both and fixed them on 22 Nov 2026.
What to Ask Next About AI Visibility
Readers who get this far usually ask one of these next.
Why did AI engines pick a competitor over me?Run your buyer questions across 8 engines and see which sources the answers leaned on. AI Answer Gap →
What should I fix on my own site first?A 16-question audit across technical, content, authority and structure signals, and what your score means. AI visibility audit checklist →
How does my sector compare?See which fintech brands ChatGPT and Perplexity cite across 4 verticals, and what they do differently. Fintech AI visibility benchmark →
Your Next Move: Read Your Visibility From the Buyer’s Seat
Copy the sheet and start a clean, repeated sample this week. Within a few days you’ll know which questions you own, which you share, and which you never appear in.
Then the real work starts: finding the pages and third-party sources the engines trust for your category, and getting your brand into them. That’s what our GEO practice does. It’s the same approach that made Vance the authoritative answer in Google AI Overviews for IMPS, UTR and payment tracking queries.
An AI visibility tracker monitors whether answer engines mention your brand by sending buyer prompts and logging the responses. You must treat these outputs as a directional repeated sample rather than an absolute measurement because engines personalise, localise, and vary from run to run. On 22 Nov 2026, upGrowth Digital tested its free tool, AI Answer Gap, against a signed-in browser across 10 checks and found the two methods agreed in only 6 of 10 checks (60%). To read your dashboard accurately, you need to account for 3 factors:
Personalisation bias from logged-in account histories altering the shortlist.
Location leaks where the engine tailors results to the searcher's physical city.
Run-to-run variance that requires repeating prompts before calling a trend.
Understanding how these 3 variables skew your data reveals why two honest tracking methods can fail in completely opposite directions.
A neutral API testing context queries answer engines through official developer endpoints without user logins or granular city coordinates. Stripping away personal session data gives every run the same neutral starting point, instead of an answer shaped by who is asking and where. When upGrowth Digital compared official APIs against a signed-in browser sitting in Pune across 5 category questions, the results diverged sharply:
Google AI Mode named the brand in 0 of 5 API answers, whereas the signed-in browser named it in 2 of 5 and inserted a local section for agencies in Pune.
Perplexity named the brand in 2 of 5 API answers, while the signed-in app named it in 0 of 5 because it tailored the response to the user's consulting positioning.
Browser bots running on a $30 a month plan inherit account history that distorts real buyer visibility.
Seeing how personal sessions flatter or bury your brand makes choosing the right tracking architecture a critical decision.
While a basic tracker logs whether your brand appears in an answer, Generative Engine Optimization (GEO) uncovers which external pages and third-party sources the engines trust so you can secure placements inside them. Knowing your win rate across 50 buyer questions is useless unless you diagnose why an engine preferred a competitor over your domain. At upGrowth Digital, the work looks like this:
Vance became the authoritative answer inside Google AI Overviews for IMPS, UTR, and payment tracking queries.
AI Answer Gap evaluates buyer questions across 8 engines to expose the specific citations driving competitor recommendations.
A 16-question audit checks technical, content, authority, and structure signals to prioritise on-site fixes.
Moving from passive scorekeeping to active source analysis raises the question of which tracking method best supports your initial baseline.
Comparing an official API workflow against a $30 a month browser bot comes down to eliminating session contamination and avoiding terms-of-service grey zones. While a browser bot types prompts directly into logged-in consumer apps like ChatGPT, Perplexity, and Google, an API script queries the models from a clean, repeatable state. In upGrowth Digital's 10-pair test, the API and signed-in browser agreed only 60% of the time. Weigh these operational differences before building your stack:
Login and history isolation: The AEO Tracker Kit runs via official APIs with no login bias, whereas logged-in bots absorb your past searches.
Geographic neutrality: API calls restrict location data to the country level, preventing your team's physical city from skewing national queries.
Compliance and cost control: You plug your own OpenAI, Perplexity, or Gemini API keys directly into the sheet and pay providers without scraping interfaces.
Examining the exact query-by-query test results illustrates just how severely a browser session can mislead your team.
A daily tracking sheet and a diagnostic audit engine solve two distinct halves of the AI visibility lifecycle. The free AEO Tracker Kit Google Sheet gives you an automated baseline across up to 50 buyer questions using strict literal name matching, while AI Answer Gap investigates the underlying citations when you lose a query. Used together, they cover 3 layers:
Daily API sampling: The free Google Sheet uses your own API keys to log clean yes-or-no presence without form gates or vendor markups.
Strict grading rules: The sheet counts a brand only when its name literally appears in the answer, and since 22 Nov 2026 AI Answer Gap counts a competitor only when its name literally appears.
Competitor citation mapping: When the sheet shows a miss, running prompts across 8 engines in AI Answer Gap reveals which third-party lists you need to join.
Looking closely at the specific test data from Pune shows why strict verification rules are necessary in the first place.
The 22 Nov 2026 experiment by upGrowth Digital showed that checking from a signed-in browser can flatter a brand on one engine and bury it on another. Across 5 non-branded category prompts and 2 engines (10 pairs with 1 run each), the API and a signed-in browser in Pune agreed only 60% of the time. The query-level breakdown highlights this opposite failure mode:
Google AI Mode localisation: The API named upGrowth Digital in 0 of 5 answers, yet the signed-in browser named it first for 'best growth marketing agency in India' and 5th for fintech performance marketing, even adding a 'Notable Agencies in Pune' section that a buyer in Mumbai or Dubai wouldn't get.
Perplexity account tailoring: The API named the agency in 2 of 5 answers (first for growth marketing and top 3 for fintech), while the signed-in app named it in 0 of 5 and opened one reply referencing the user's own 'growth-consulting positioning'.
These conflicting outcomes exposed deeper scoring flaws inside the automated grading software itself.
Auditing the internal grading pipeline of AI Answer Gap on 22 Nov 2026 found 2 grading issues that skewed results. By inspecting raw logs across 25 answers and 20 completed runs, upGrowth Digital found and fixed 2 issues:
Hallucinated competitors in 6 of 25 answers: The AI judge tasked with listing named brands copied rival names directly from the tracking prompt list even when they never appeared in the engine's response, artificially inflating the 'who beats you' chart.
Penalising empty runs in 7 of 20 records: When Google displayed no AI Overview at all, the system logged it as 'not named', dragging total visibility scores down by 0.6 points on average and 1.5 points at most.
The fixes: A brand now counts only if its string literally exists in the text, and empty responses are excluded and tagged 'No AI answer'.
Fixing tool accuracy is the prerequisite to building a broader strategy that turns visibility into real category authority.
Deploying the free AEO Tracker Kit gives your team an automated, neutral sampling setup in Google Sheets using official APIs rather than fragile browser scrapers. Created by upGrowth Digital with literal name matching and no form gate, the sheet lets you pay API providers directly while keeping full control of your data. Follow this 4-step implementation sequence:
Duplicate the template: Make your own copy of the AEO Tracker Kit sheet, which automatically bundles the underlying automation script.
Configure your competitive landscape: Enter your brand name, domain, target competitors, and up to 50 non-branded buyer questions.
Connect official developer keys: Paste your OpenAI, Perplexity, or Gemini API key into the script settings; any engine without an active key is cleanly skipped.
Execute or automate sampling: Trigger a manual pass from the AEO Tracker menu or schedule a recurring daily run to build a longitudinal dataset.
Once your daily runs start populating, you must apply strict analytical rules so you do not misinterpret run-to-run fluctuations.
To keep your AI visibility tracker from turning into a misleading vanity metric, your team must pair repeated neutral sampling with manual quality control and hard revenue attribution. Because answer engines vary from run to run, upGrowth Digital configured AI Answer Gap to execute every question 3 times internally and track flip frequency. Implement this 4-part verification protocol across your team:
Enforce multi-day repetition: Run each buyer prompt at least 3 times across several days before declaring an upward or downward trend.
Quarantine manual browser checks: Never test from a personal session; if you must check by hand, open a logged-out private window so your city and history do not leak in.
Conduct a 5-answer spot check: Manually compare 5 raw engine answers against what your software recorded to verify the grader did not invent competitors.
Tie trends to commercial outcomes: Correlate visibility gains directly with branded search volume, AI referral sessions, and pipeline leads.
Connecting these metrics prepares your organisation to adapt as answer engines intensify their personalisation algorithms.
A 60% agreement rate between official APIs and signed-in browsers suggests, in upGrowth's view, that single-point checks are unreliable for answer engines and that reporting should move toward probabilistic share-of-voice reporting. In upGrowth Digital's 22 Nov 2026 benchmark across Perplexity and Google AI Mode, personalisation and location leaks caused methods to fail in opposite directions (agreeing in only 6 of 10 checks). To future-proof your reporting and budget allocation, consider 3 adjustments:
Retire deterministic rank snapshots: Educate executives that an AI visibility tracker delivers a sample, meaning reports must show win rates across 3 repeated runs over multiple days.
Reallocate budget from tracking to source capture: Use free API sheets like the AEO Tracker Kit for baseline monitoring, freeing capital to get your brand cited on the third-party lists AI engines actually read.
Anchor KPIs in bottom-funnel signals: Judge your GEO investments by growth in branded search, AI referral sessions, and qualified leads rather than raw mention counts.
Shifting budget toward third-party citations leads directly into how you should systematically audit your sector's competitive landscape.
Because engines like ChatGPT and Perplexity synthesise answers from trusted external directories and peer reviews, passive dashboard monitoring will never close your visibility gap. Your competitors often write or influence the exact lists that AI models read. In upGrowth Digital's view, the move from monitoring to action has 3 stages:
Benchmark sector citation patterns: Analyse how engines cite leaders across 4 fintech verticals or your specific niche to see what top brands do differently.
Map competitor source dependency: Run up to 50 buyer prompts across 8 engines in AI Answer Gap to isolate the exact third-party pages recommending your rivals.
Execute a 16-question on-site audit: Fix foundational bottlenecks across technical, content, authority, and structure signals so engines validate your domain when they crawl your ecosystem.
Before you can execute this source-capture strategy, however, you must eliminate the common tracking mistakes that blind teams to their true baseline.
Amol has helped catalyse business growth with his strategic & data-driven methodologies. With a decade of experience in the field of marketing, he has donned multiple hats, from channel optimization, data analytics and creative brand positioning to growth engineering and sales.
Your buyers ask AI before they search. Run AI Answer Gap and see whether ChatGPT, Perplexity, Gemini and Google AI Overviews name your brand for the questions that matter in your category.