Summary: As the CTO of a software start-up, Joe sits on both sides of the software purchase decision. In this blog, he shares three questions he asks his technology vendors but rarely sees from potential clients, and what the answers reveal.
By Joe Mosby, Chief Technology Officer, Altvia
Back in the early 00s, technology-enabled firms expected to have their hardware and software installed in a data center that they controlled, with all their security parameters set in place. It was capital-intensive, sure, but that was all that was available. Around 2006, “cloud computing” entered the marketplace, followed by “software as a service,” and all of a sudden, things weren’t as clear-cut. You could get great software at a compelling price, with no infrastructure to maintain, but you had to outsource a lot more of your security and compliance practices. Firms had to adjust all of their due diligence practices accordingly, and buying practices had to shift.
The age of AI is bringing on a similar shift. It’s given firms new possibilities in workflows, and it offers the tantalizing possibility to do more without all the same engineering requirements. Yet buying practices haven’t kept up. Due diligence questionnaires are still stuck in the cloud computing era, where data residency was often the primary concern. Now you’ve got a whole stack of third party models that can start to make business decisions on your behalf. Questionnaires aren’t keeping up with the risks and opportunities this provides.
I’m on both sides of the software purchase decision. As the CTO of a software startup in a security- and compliance-sensitive industry, I’m the person talking to our vendors to identify risks to our business and our clients’ data. And in that same role, I’m also the one going through all of our inbound due diligence questionnaires from clients looking to protect their own businesses.
The questions I ask my vendors are not the same as the ones that I’m asked. Here are the top three questions I’d recommend be included in any due diligence questionnaire, and why they matter.
“Do you train models on my data?”
In 2017, vendors regularly trained models on customer data – that was the point. If contextual intelligence wasn’t plugged into the model, there was no way it was going to make a good decision. However, in 2026, the only firms training are big names like Anthropic, OpenAI, Google, or Microsoft, household names in the space. All of the rest of us are using one of the excellent models provided by the frontier labs, because training is expensive.
The way in which we use those models can be very different, though. At Altvia, we have contractual agreements that prevent our AI vendors from using any of our business data for training purposes, whether it’s client-associated or not.
While I’m regularly asked if I train models on customer data (no, because I don’t train models), I have only been asked once if I have contractual agreements with my subprocessors that prevent them from doing the same.
So, when I ask my own vendors, I ask: “are you using any third-party AI vendors to work with customer data, and do you have agreements to protect our data with those vendors?”
“Can I customize any AI workflows?”
When I was a kid, I grew up watching Disney’s Fantasia. Mickey Mouse re-enacts The Sorcerer’s Apprentice, where a young apprentice takes over his boss’s spellbook and tries to invoke spells to do his chores for him. It ends poorly, and the sorcerer has to come back to rescue his apprentice.
For the past decade, software providers have done a marvelous job at providing tools in the form of APIs, and many are embracing MCP as well. Prior to the age of AI, there was a natural filter on the use of these APIs – you had to have an engineer involved. You had an expert who needed to scrutinize what needed to be done, and that expert paused to ask questions when they were confused.
Now, Claude/ChatGPT are more than happy to let you customize to your heart’s content. You’re a wizard with a new spellbook who can start chanting incantations while new products emerge. But the question pops out: are you the sorcerer? Or the sorcerer’s apprentice? I’m a sorcerer in some things, and an apprentice in others. (Marketing still won’t give me the keys to ad spending.) When I’m not a master myself, I need guardrails.
So, I assume I can customize any AI workflows with any product, because everyone has APIs that I can hook Claude into. I’m more concerned about the guardrails. What’s going to prevent me from causing unintended side effects? Do I have auditability over my actions? And can I trust that those guardrails will work for my team as well?
The question, then, is not “can I customize AI workflows?” but “what systems and guardrails are in place so that I can reduce risk when I customize AI workflows?”
“Does your AI tooling account for my industry?
My last three roles could not be more different. I worked with Fortune 100-scale accounting deployments, where it could reasonably take hours to run a sufficiently complex query over enough data. I’ve worked with payments providers with thousands of transactions a second that all looked mostly the same. Here at Altvia, we have tons of industry nuance that needs to be factored into every decision.
Yet all of those roles get rolled up into “software as a service.” They couldn’t be more different, even for software people!
If I approached the business of private capital in the same way I did high-velocity e-commerce payments, it wouldn’t work. Transactions in e-commerce are mostly uniform, while transactions in private capital have substantial complexity. If I’m designing an AI solution, I can’t have them look the same way. I need to be thoughtful that my tooling takes industry context into account, because making a mistake (like missing a side-letter obligation) could have dramatic consequences.
Our clients will have a better product when AI tooling takes into account the specifics of their business, and mindfully considering how it factors into their work without needing to manually train on individual data. Everyone in private capital has side letters to worry about. If I don’t build my software to think about those industry-specific considerations, then my customers are getting a subpar experience.
When I’m doing business with my vendors, I’m asking about the difference in their tooling for me versus anyone else in the space. I would love to see more firms doing the same.
Better Questions Lead to Better Outcomes
Due diligence is a part of life. But AI has upended its traditional forms. We don’t know how to ask the right questions about security and compliance, and we’re not digging in enough on the capabilities that this new technology enables for us. So I hope my customers steal my homework and start adding these questions to their due diligence forms. They’ll have a better understanding of what they’re getting and how it works. And the first time all three cross my desk, I’ll be ready to answer for Altvia.