By: Ethan Rogers
Most of the IT industry’s AI announcements over the past two years have followed one shape: take an existing product, add an AI layer, and call it AI-native. Rob Smith, a former Gartner analyst who founded Lionfish Tech Advisors in 2021, is building closer to the opposite direction. His firm’s products are AI-powered, but the pitch is not the AI. It is the roughly thirty former Gartner analysts who check what the AI produces before a client ever sees it.
That is a deliberately unfashionable position in a market where AI-native is a selling point on its own. Here is why Smith thinks the framing is backward for high-stakes buying, how the human layer actually works inside Lionfish’s products, and what it means to market people as the differentiator in an AI cycle.
Key Takeaways
▪ Lionfish builds human verification into the data pipeline, not as a support bolted onto an AI product.
▪ Smith’s failure mode for rushed AI is the oldest rule in computing: garbage in, garbage out.
▪ The Pyramid rating updates at least daily, several times a day in fast categories, versus a yearly chart.
▪ Human verification is slower and caps market coverage; Smith treats that discipline as a feature.
▪ Ask any AI advisory tool what happens to the data before the AI touches it, and who is accountable.
The Industry’s Default Pitch, And Why Smith Is Skeptical
Smith does not dispute that AI is reshaping IT work; he says it has changed his industry over the past two years more than almost anything before it. His skepticism is narrower. He thinks adoption is moving faster than data quality can support, and that a lot of organizations are learning that the hard way. His shorthand is the oldest rule in computing: garbage in, garbage out. A model is only as reliable as what it was trained on, and much of what is available to train on, marketing pages, incentivized reviews, and outdated documentation, was not reliable to begin with.
He expects the problem to correct itself, not because models get dramatically smarter, but because organizations get more disciplined about what they feed them. Most deployments today, in his description, are feeding the system everything in the kitchen sink instead of the information that matters. He thinks the market is moving toward more targeted, curated data, and Lionfish is built to be ahead of that shift rather than reacting to it later.
What Does Adding The Humans Back In Actually Look Like?
The human layer is not a support team bolted onto an AI product. It is built into the data pipeline. Every fact that flows into the Aquarium, the firm’s competitive-intelligence platform, is checked by a former Gartner analyst before the AI layer can use it to generate a recommendation. The internal shorthand, from colleague Michael, is blunt: use AI, but verify. That step is the entire premise of Periscope, which produces instant buying recommendations on the same human-checked foundation.
The clearest example is the real-time rating system Lionfish built as an alternative to the once-a-year chart, the Pyramid. Instead of a report that is already stale by the time it clears review, the underlying data updates at least once a day, and several times a day in categories moving as fast as agentic AI. That cadence is only possible because verification is built into the pipeline from the start, not added as a final check before publication.
This Is Not A Cost-Free Choice
Human verification is slower and more expensive than letting a model run unsupervised on public data, and it puts a hard ceiling on how many markets Lionfish can credibly cover, since the firm can only move as fast as its analysts can check. Smith treats that constraint as a feature. It is the same discipline that made analyst research trustworthy in the first place, applied at a faster cadence than a once-a-year chart allows.
Why Is This A Harder Story To Tell Than “We Use AI”
Brad LaPorte, a Lionfish colleague, is candid that the firm’s biggest challenge is not the product; it is the narrative. “We’re quietly awesome,” he says, “but it’s getting through that narrative” that has been the hard part. That is common for companies whose edge is process discipline rather than a flashy feature. It is easier to market “we added AI” than “we added rigorous human oversight to AI,” even when the second claim is the one that protects a buyer from a bad decision.
For an IT leader evaluating tools right now, that distinction is worth attention well beyond Lionfish. The question to ask any AI-powered advisory tool is not whether it uses AI; nearly everything does. It is what happens to the data before the AI touches it, and who is accountable for catching the mistake before it reaches a purchase order.
Frequently Asked Questions
What Does Human-In-The-Loop Mean At Lionfish?
Every fact entering the Aquarium platform is checked by a former Gartner analyst before the AI layer uses it, so verification sits inside the data pipeline rather than as a final review or a support desk.
Why Does Data Quality Matter More Than Model Quality Here?
Because a capable model still returns a wrong answer if the underlying facts are wrong. Smith’s shorthand is garbage in, garbage out, and his fix is to curate and verify the inputs first.
What Is The Pyramid?
The Pyramid is Lionfish’s real-time vendor rating system, an alternative to the once-a-year chart. Its underlying data updates at least daily, and several times a day in fast-moving categories.
The Bottom Line
Adding AI is easy, and everyone is doing it. Adding accountable human judgment underneath the AI is slower, more expensive, and harder to market, which is exactly why it is the part worth checking for. Lionfish’s bet is that when a real purchase is on the line, buyers will pay attention to who verified the data, not who wrote the algorithm.
That same human-verified pipeline underpins both the firm’s core platform and Periscope, its buying-recommendation tool.
