Data Sovereignty, Governance Muscle, and Agentic AI Liability

Greg

G’day, and welcome back to the Deep Dive. Greg here on the Gold Coast, and I’m joined, as always, by Liliana up in Brisbane. How you going, Lili?

Unknown Speaker

I’m good, I’m good. Pepper’s finally asleep, so if we don’t hear barking, we’re winning. Today’s topic is huge, though. You’ve been calling it a bit of a riddle, right?

Greg

Yeah, let’s kick off with that. When a massive, world‑changing technology like AI comes along… who actually adapts to it fastest? Most people will say, “Oh, the startups. The scrappy, move‑fast‑and‑break‑things crowd.”

Unknown Speaker

That’s the default story we all grew up on, right? Two people in a garage, shipping code overnight, disrupting the dinosaurs.

Greg

Exactly. But what we’re seeing right now is almost the opposite. Some of the most heavily regulated industries on the planet – traditional retail banks, big insurers, those massive incumbents – are actually ahead on safe, effective AI adoption.

Unknown Speaker

Which sounds completely backwards at first. We usually blame regulation for slowing everything down. All that red tape, compliance, lawyers… it feels like the enemy of innovation.

Greg

Right, but in the sources we’ve been working with, the experts talk about something they call the “governance muscle.” And that flips the story. Those big, regulated firms have spent decades building internal machinery to handle risk – committees, compliance teams, audit trails, tight access controls.

Unknown Speaker

Yeah, if you think about a major bank for a second, they’re sitting on highly privileged customer data. They’ve already invested billions in systems to lock that down. People are trained on very rigid boundaries: who can see what, what gets logged, what triggers a review.

Greg

So when AI turns up – which is messy, uncertain, genuinely hard to predict – they’re not starting from zero. They can plug this new technology into an existing risk‑management machine, instead of making it up as they go along.

Unknown Speaker

Compare that to, say, a 10‑person startup. They might be able to ship a new AI feature over a weekend, but they usually don’t have a framework for even asking, “Does this breach a data‑privacy law in Europe? Are we accidentally leaking client information?”

Greg

Exactly. The bank has people whose entire job is to think about that. So their “governance muscle” lets them experiment with AI in a more contained way – sandboxed environments, permissions, red‑team reviews – which means when they do roll something out to customers, they can do it with a lot more confidence.

Unknown Speaker

But there’s a really interesting tension in the report. On the one hand, executives are incredibly bullish on AI – around 80% saying they plan to increase investment. On the other hand, more than 60% are struggling to see any positive return so far.

Greg

That’s the widening ROI gap. Everyone’s excited, capital is flowing in, but the value isn’t showing up on the P&L yet. A lot of organisations are stuck in what the report calls the “experimentation phase.”

Unknown Speaker

You see it on the ground. They’ve bought the licenses, maybe rolled out an internal chatbot, given staff a tool to draft emails or summarise documents. People play with it for a while, it feels clever… but it doesn’t really change how the business runs.

Greg

Yeah, they’ve changed the software, but they haven’t changed the workflow. And until you do that hard, unglamorous work of redesigning processes, upskilling staff, rethinking who does what, you’re not going to see real ROI.

Unknown Speaker

This is where that “opportunity bias” from the expert panel comes in. Leaders are leaning so far into the hype – chasing the opportunity – that they underweight what it actually takes to implement AI safely and profitably. They sort of assume the tech itself equals advantage.

Greg

And it doesn’t. The tech is just one ingredient. Without governance muscle and serious change management – training programs, new KPIs, sometimes even restructuring – you end up with a very expensive toy sitting on top of an unchanged organisation.

Unknown Speaker

The sources even suggest there’s a clock ticking. Roughly a 12‑month window for companies to move from “cool proof‑of‑concept demo” to “tangible, defensible results.” Boards and investors are going to start asking, “What did we actually get for all this AI spend?”

Greg

So the riddle answer is: heavily regulated banks and insurers can look slow from the outside, but because they’ve built that governance muscle over decades, they’re surprisingly well placed to cross that gap – to go from experimentation to real value – without crashing the car.

Unknown Speaker

And the startups? They still have amazing speed, but with much higher downside risk if they stumble into a data leak or a regulatory mess they never even saw coming.

Unknown Speaker

So, let’s pivot to this brutal new reality for software companies. One of the stories in the material really stuck with me – the LinkedIn example.

Greg

Yeah, this is wild. The founder of LinkedIn used an AI coding tool – Replit – to essentially build a fully functioning, AI‑generated replica of LinkedIn. Not as a multi‑year engineering project, but basically in an afternoon, with minimal coding effort.

Unknown Speaker

Just let that land for a second. You’ve got a platform that originally took huge teams, years of work, hundreds of millions of dollars… and now an AI can clone the core functionality orders of magnitude faster and cheaper.

Greg

If you’re running a software business, that’s a bit terrifying. It forces you to ask: if code is that easy to replicate now, what exactly is your competitive moat? Because AI is collapsing those historical advantages: big engineering teams, massive upfront capital, time to build.

Unknown Speaker

Startups can now test product‑market fit at lightning speed. Instead of having to raise a big round just to get to a prototype, they can lean on AI to write code, launch something quickly, and iterate almost in real time.

Greg

So if software itself is becoming commoditised – almost like a cheap, abundant resource – the experts in the sources argue that the defensible moat shifts somewhere else. And that’s where data sovereignty comes in.

Unknown Speaker

Data sovereignty in this context is basically: who owns, controls, and can legally use which data, and where it’s stored and processed. And the big incumbents have a trump card startups don’t – deep, proprietary, historical, sector‑specific data.

Greg

Exactly. A generic AI model that anyone can subscribe to for twenty bucks a month is like hiring a brilliant intern on day one – super smart, but zero context about your customers, your supply chain, your past decisions.

Unknown Speaker

But if you take that generic model and you hook it up to your secure vault of data – decades of customer histories, internal performance metrics, all those logged support tickets – suddenly it’s not a generic intern anymore. It becomes a specialist in your business.

Greg

And that specialist knowledge is not something a rival can clone overnight. They might copy your user interface, they might even copy a workflow, but they can’t recreate twenty years of cleaned, structured, permissioned data about how your particular market behaves.

Unknown Speaker

So in this new landscape, the report is pretty blunt: your long‑term value isn’t really the software you built. It’s the proprietary data you hold, and how intelligently – and safely – you let AI learn from it.

Greg

But the moment you start talking about data as the moat, you run head‑first into regulation. And the global regulatory map right now is completely fragmented. There is no single rulebook.

Unknown Speaker

Yeah, the report basically takes us on a tour. You’ve got China at one extreme: the strictest environment, with very specific, binding laws around generative AI, deepfakes, even AI companions. The motivation there is maintaining tight control over the domestic information ecosystem and limiting social risks.

Greg

Then you jump to Europe, with the EU AI Act. Different philosophy: they’re trying to build public trust by putting risk‑based safeguards in place before products go to market. But that’s drawing criticism from some in the tech sector, because the compliance burden for a small or medium enterprise could actually be higher than the cost of building the AI in the first place.

Unknown Speaker

Which raises that fear that the EU might clip its own innovation wings by making it too hard and too expensive to comply, especially for smaller players.

Greg

Then you look across Asia – excluding China – and you see almost the opposite. Japan is reviewing copyright and privacy rules to remove obstacles to AI development, trying to make data use easier, not harder.

Unknown Speaker

Singapore is leaning heavily on principles and voluntary guidance instead of tight, binding laws. And India is going very targeted: focusing mainly on specific high‑harm areas like election misinformation rather than regulating the whole AI stack.

Greg

Sitting here in Australia, we’re in that “middle power” category the panel talked about. We don’t control the foundational models – that’s largely the US and China – so we have to think strategically about sovereign AI: our own data, our own compute, our own standards, so we’re not completely dependent on foreign tech stacks.

Unknown Speaker

Because if you rely entirely on infrastructure controlled somewhere else, and there’s a trade dispute, or a terms‑of‑service change, your critical services – banking, health, even government – could be disrupted overnight. That’s a genuine national‑security and economic‑independence issue.

Greg

So at both the national and corporate level, the strategy is starting to converge: software is easy to copy, but data is hard to copy and politically sensitive. Whoever can hold and govern the right data, in the right jurisdiction, with the right safeguards, holds the real power.

Unknown Speaker

And for businesses operating across borders, that means your AI roadmap isn’t just a tech roadmap anymore. It’s also a regulatory and geopolitical puzzle, with completely different expectations in each region you touch.

Greg

Let’s move to where the report spends a lot of time: agentic AI. Most people’s day‑to‑day experience of AI so far is generative – you type a prompt, it gives you text, code, images. It’s powerful, but fundamentally passive.

Unknown Speaker

Agentic AI flips that. These are systems designed to go out into your digital environment, make decisions, and take actions without asking you to click “approve” every five seconds.

Greg

Yeah, imagine this in a really concrete workflow. Instead of an AI that just drafts an email for you to review, you’ve got an AI agent that reads your inbox, notices a vendor is late on a shipment, decides that breaches the contract, drafts the cancellation notice, and then actually executes the termination on your behalf.

Unknown Speaker

Or an agent plugged into your supply chain that doesn’t just suggest an optimisation, but goes ahead and rewrites configurations, changes supplier allocations, and pushes those updates into live systems. That’s a very different risk profile from a chatbot helping with PowerPoint slides.

Greg

And that’s where the legal ambiguity kicks in. The sources talk about Singapore’s approach, which I find really elegant – they’re leaning on what they call the “employee analogy.”

Unknown Speaker

Basically: treat what the AI agent does the same way you’d treat what a human employee does. If your staff member accidentally leaks client financial data, the company is still on the hook. You don’t get to blame “the employee” in a way that removes corporate responsibility.

Greg

So with agentic AI, you also don’t get to say, “Oh, the software made a mistake, not us.” Regulators are pushing back on that. You can’t use AI as a legal shield.

Unknown Speaker

The report calls this the “many hands” problem. If an AI agent auto‑optimising your logistics writes dodgy code that crashes a supplier’s system and causes millions in damage, who’s actually liable? The vendor that built the model? The employee who wrote the prompt? The person who integrated the tool?

Greg

The panel – especially Professor Mimi Zhu – basically says corporate governance already has a clear answer. Even if the tech is new, the principle isn’t: accountability ultimately rolls up to the board and senior management. You can delegate tasks to software, but you cannot delegate accountability.

Unknown Speaker

And here in Australia, they’re even exploring a formal digital duty of care, to make that explicit. So if you’re a director, or even just a team leader, you can’t cross your fingers and hope the agent behaves. You have to actively manage the risk.

Greg

That’s where those ex ante measures come in – preventative steps you take before something goes wrong. The report lays out a kind of checklist.

Unknown Speaker

First, independent audits of AI systems before they ever touch live data or real customers. Not just a quick internal test, but rigorous, documented review of how the system behaves, where the guardrails fail, what biases or failure modes might exist.

Greg

Second, contractual risk‑shifting – what the sources call “deep pockets legal counsel.” You want watertight contracts with your AI vendors spelling out exactly who is responsible if their model hallucinates, misbehaves, or exposes you to regulatory breaches.

Unknown Speaker

Third, stress‑testing or penetration testing for AI – and I really like this analogy to cybersecurity. Just like you try to hack your own network to find the flaws, you should be trying to break your own AI agents before the public ever goes near them.

Greg

The sources point to Singapore’s AI Verify Foundation as an emerging example of how those testing standards might work. If you skip that kind of rigorous testing and something goes wrong, you’re going to look incredibly negligent in a courtroom.

Unknown Speaker

But there’s another layer here that often gets ignored in these conversations: people. The humans actually sitting at the desks using these tools. The report talks about a wave of job anxiety in sectors like finance, law, consulting – places that are already seeing AI‑linked restructures.

Greg

Yeah, if your staff quietly believe that every new AI tool is basically training their replacement, they’re not going to lean in and use it. You won’t get the experimentation, the honest feedback, the creativity you need to scale the technology.

Unknown Speaker

So leadership has this really delicate task: push hard for productivity and competitive advantage, while still maintaining psychological safety. People need to feel like they can learn, try things, even make mistakes with these tools, without it becoming a countdown clock on their job.

Greg

And all of this is happening with a technology that even the creators describe as a bit like a genie or magic. Large language models work by absorbing billions of patterns, not following neat, human‑written rules. So even top engineers can’t trace exactly why the model gave a specific answer in a specific moment.

Unknown Speaker

Which makes it a black box for boards. How do you govern something inherently probabilistic, especially when you didn’t build it yourself and you’re licensing it from a third party?

Greg

For most organisations, that’s the core risk: not that the underlying model is secretly evil, but that they don’t have enough internal capability – legal, technical, operational – to understand and control what happens when that black‑box genie is plugged into their highly sensitive data.

Unknown Speaker

So if we pull it all together: first, build your governance muscle so you can experiment quickly without breaking the law. Second, recognise that your real moat isn’t your code – it’s your proprietary data and how responsibly you use it. And third, as we move into the era of agentic AI, keep reminding yourself: accountability remains human.

Greg

You can’t outsource responsibility to the agent, no matter how clever it is. If you choose to deploy it, you own the outcomes – good and bad.

Unknown Speaker

And maybe the big, provocative question to sit with is the one from the sources: if AI becomes a cheap, commoditised genie that anyone can access, do the mega‑corporations of the future just end up being the ones that quietly hoarded the best, most obscure historical data?

Greg

If software is cheap, data is gold. So as you’re listening, think about where the “gold” actually is in your world – and whether you’ve got the governance muscle to handle it.

Unknown Speaker

Alright, that’s all we’ve got time for today. Greg, as always, loved this chat.

Greg

Same here, Lili. Thanks for lending us your ears, everyone.

Unknown Speaker

Keep asking the hard questions, take care of your people as you experiment with this stuff, and we’ll catch you on the next Deep Dive.

Greg

Cheers.