Shadow Sciences Group Warns of Human Oversight Gaps as AI Automates High-Impact Actions

17 September 2026 | Thursday | News

Risk advisory firm highlights the need for human approval in irreversible workflows as agentic AI moves beyond recommendations to executing financial and business actions
Picture Courtesy | Public Domain

Picture Courtesy | Public Domain

The standard advice before sending a wire transfer is to phone the recipient first, on a number you looked up yourself rather than one printed in the email. It is good advice. It also depends entirely on somebody being there to place the call.

Shadow Sciences Group, a boutique exposure intelligence and risk advisory firm, says that somebody is disappearing from ordinary business workflows faster than anyone is replacing what they were there to catch.

Every verification control assumes someone stops to check. Automated systems are removing that moment.

The firm's August series "Who Decides" traced how judgment became something people hand off, moving from labor to research and eventually to the conclusion itself. Its argument now is that the handoff did not stop there. It carried on into the action.

"A recommendation still leaves somebody in the loop, even a distracted one," said Kia Hakimi, Partner. "There is a click. That click is where our controls live. It is where you catch that the account number changed. When the system acts instead of suggesting, the click is gone. Nothing got defeated. The control just has nowhere left to attach."

In April, Australia's Cyber Security Centre published guidance on agentic AI together with CISA, the NSA, Canada's Cyber Centre, and the national cyber agencies of New Zealand and Britain. Two of its recommendations bear on the firm's argument.

The first asks organizations to sort automated actions by impact, likelihood and reversibility before deciding how to guard them, and the document closes by ranking reversibility ahead of efficiency gains.

"That is the whole test," Hakimi said. "Not importance. Reversibility. Nobody needs to review a calendar invite. If the wrong answer wires money overseas or records a document against your title, somebody has to be standing there."

The second recommendation states that decisions about when a human must approve something belong to the people who design and operate the system, and cannot be handed to the system itself.

"Six governments sat down and wrote that sentence, which tells you it was already happening," Hakimi said. "Your reviewer is looking at the exceptions that some other system decided were worth showing them. So the judgment about whether the judgment held up got delegated too. At some point you are not really supervising it. It is just telling you that everything went fine."

The expense comes from what cannot be reversed. FBI figures put business email compromise at $3.047 billion across 24,768 complaints last year, second only to investment fraud, with roughly 86 percent of the money moving by wire or ACH, where recovery is least likely. Of that total the bureau attributed more than $30 million specifically to AI-enabled schemes, which works out to about one percent and is the first year the figure could be measured at all.

Shadow Sciences is not arguing against automation, which it calls neither achievable nor desirable to avoid. Its position matches the advisory's, which is to automate the things you could undo and keep a person on anything you could not.

 

Survey Box

Vote for the most influential trend in 2025:

What's Driving the Future of FinTech?v

× Please select an option to participate in the poll.
Processing...
× You have successfully cast your vote.
 {{ optionDetail.option }}  {{ optionDetail.percentage }}%
 {{ optionDetail.percentage }}% Complete
More polls
Stay Connected

Sign up to our free newsletter and get the latest news sent direct to your inbox

Fintech Business Asia, a business of FinTech Business Review
© 2026 FinTech Business Review. All Rights Reserved.

Show

Forgot your password?

Show

Show

Lost your password? Please enter your email address. You will receive a link to create a new password.

Back to log-in

Close