In Conversation with Dr. Andrew Forde at Web Summit: AI Ethics, Governance, and Real-World System Design

Dr. Andrew Forde is a Partner in KPMG Canada’s Technology Strategy and Digital Transformation practice and the firm’s inaugural Head of AI Research. He earned a PhD in Industrial Engineering and Data Sciences from the University of Toronto, after completing a Master’s degree in Mechanical Engineering and Data. Andrew started his career as a NASA research scientist. A classical violinist, Andrew has performed for Presidents Bill Clinton and George W. Bush, performed with Justin Bieber, Sting and Pitbull, and has composed for various films & commercial projects.

Since I’m in the presence of such an accomplished violinist, I must ask: what’s your favorite song to play?

I would say Liebisfreud by Fritz Kreisler because it’s both beautiful and exciting. I came onto Fritz Kreisler when I was maybe 12 or 13 and it was foundational in renewing my appreciation for the art form. Joshua Bell and Paul Crocker have an album together where they play a bunch of Fritz Kreisler’s music, and that album was everything for me for a long time. Secondly, I would say “Autumn Leaves” by Joseph Kosma. It’s a jazz standard that has a lot of freedom to be expressive. 

[Me: Your answers are way better than mine. I humbly volunteer the string rendition of Toxic by Britney Spears—it’s haunting!]

Based on what you’ve said about how music impacts your work as an engineer, do you think humanities education should be a bigger part of engineering school?

Yeah, absolutely. It’s becoming increasingly important as we move forward. I’m a firm believer that with the direction technology’s taking, it’s going to force a return to the essence of what it means to be human. The people who are most equipped to not only deal with the change, but also deal with the impacts of the change, are going to be people who understand the technology deeply. And I mean that meaningfully, as in, if I open up the code bank, can you look at it and tell what’s going on? Now infuse that same person with a philosophical-classic literature-art understanding of what the human spirit represents. The marriage of those things will be incredibly important. 

[Me: Yeah, the whole “move fast and break things” never built in our humanity. It was just like, ah, it’s fine, fine—disruption! money!—and now, 20 years later, we’re in a technofascism-feudalism kind of world]

The one thing that has also been playing in the background, amidst what you just described, is the need or the feeling to oversimplify things. For a long time, if you couldn’t articulate complicated concepts—simply—people assumed you didn’t know what you were talking about. We’re at a moment now where it’s like, no, you probably need to be able to elevate yourself to the complicated concepts that are now present and you can’t just simplify it. Nor should you. You’re losing a lot of important information if you just truncate. 

[Me: It drives me mental when people are like, “oh, it’s not that deep.” When yes, it actually is that deep. Put your hip waders on] 

Like, this is where, again, why we’re here is because people aren’t actually stopping and thinking about stuff. The humanities help with that.

Ultimately, the question that we’re gonna have to ask ourselves is: when certain things can be done better, safer, et cetera, by machines, how will we reorganize ourselves? And I never discount the human spirit in any of that. Because it’s not the first time we’ve sat at an intersection of something truly transformational. This feels a little bit different because we grew up with technology.

At this moment, anything that we’re saying will be the future is a wild ass guess. We just don’t know. But the one thing I do know is we will come out of it as people. We will organize ourselves in such a way that the meaning of our humanity—even if we can’t define it—will still figure out a way to shine through. Humans have always done that. 

[Me: We’re like cockroaches. We’re always there] 

Many enterprises are racing to adopt AI while still running on legacy infrastructure. Does modernization become riskier when AI is layered onto systems that were never designed for this level of autonomy or complexity? I am thinking of the fact that most ATMs worldwide still run on Windows XP.

The risk of modernization is that a lot of these systems—the people who set them up are no longer in those roles—or on earth. Knowing what the downstream effects will be if you change or modify something, is unknown. For things that don’t necessarily have a lot of material impact, modernize away, see what breaks, and then figure out how to fix it. For things that will have significant material impact to people, to systems, to institutions, the smarter way is to build in parallel. Then cut over when you are confident that things will be okay. You don’t want to take that risk in an OR, right? Don’t try to undo and re-engineer the legacy. Start fresh. 

Do you think organizations still treat ethics as a governance layer above technology, rather than something embedded directly into system infrastructure and design decisions?

This is an interesting question because I don’t know that I walk into many rooms where ethics is a discussion point at the table. 

[Me: That scares me. Whole other conversation, but yikes] 

Right? What I attribute that to is, again, the humans in the room are making decisions based on their own world models, which have their ethics embedded into it. It’s not necessarily something that is treated as an independent variable, but it’s a dependent variable that’s already tied into their worldview and how they’re approaching the problem. That’s the distinction I make first. Then from that perspective, most rooms that I’m in with clients, with boards, are trying to do the right things. It’s a question of prioritizing the various competing either interests or outcomes. That ties back to requiring a cohesive mechanism to which you are assessing the decisions you’re making, whether that’s infrastructure or endpoint use. 

Generally, people do make decisions that are in the best interest of the outcome that they’re trying to optimize for. The ethical question is tied into, “what is the outcome that we’ve agreed that we’re trying to achieve?” If that’s done appropriately, then you generally have better outcomes. If not, then you find yourself in a quagmire where the thing that you’re optimizing for might not necessarily be the thing that you need or the best design decision. Then you’re at a tension point. When you face those tension points, the conversation generally has to go back to the first principles of what it is that we’re trying to achieve. 

[Me: When you mentioned competing interests, the first thing that popped in my mind was, here we go again. It’s gonna be the people with the most money and power who are deciding those priorities. Is there a way to engineer systems where the usual suspects don’t commandeer everything?] 

That’s where regulation has to play a role. Regulators need to look at the public good. 

Many companies talk about “responsible AI.” What separates responsible deployment from responsible design, beyond checklists and compliance frameworks?

In Canada, we have OSFI to regulate financial models, and it’s excellent. If we’re not being hyperbolic, we’ve been using machine learning predictive models for 20 some years. So from our financial perspective, it’s already regulated that you document how your models were made. You stress test them, you see how they behave in different scenarios. You make sure that the material impacts are calculated. You have third parties review your models every single year and document their findings. There’s already a rich environment of how we deal with models and finance that we deem materially impactful. And I think there’s a lot there that can be carried over. 

This is why I think it ties very closely with how we decide to regulate because a lot of the models that we’re using are not developed by us, or the institutions using them. And so it’s difficult to then say, well, let me see the documentation. Let me see your test. What was it trained on? And they’re not showing us that because under the Harper government, anti-circumvention laws were lobbied by the U.S., which we adopted, against a lot of pushback. It was pushed through. In CUSMA, there’s a chapter on source code that says what we can and can’t look at and all this sort of information. It’s the government’s role to look at that and ask, “is it the best interest of Canadians if we can’t check what data your model was trained on or how it’s actually making these decisions?” My hope is that if we are able to look at these things more holistically, and ask what things we need to be able to see and ask in order to get to a place where we’re comfortable. Identifying and measuring the performance and what that means for the Canadians who are using it or being affected by it. Then I think you start to have the right conversation. 

In real-world deployment, what concerns you more: technical failure, or institutional overconfidence in systems that appear more certain than they actually are?

Institutional failure. The technology works, that’s not keeping me up. If the technology development cycle stopped today, we probably have 6 to 7 years of utility to extract from it. I’m not worried about the technology side. If anything, I’m more worried about the institution. 

What does meaningful transparency look like in enterprise AI systems when even technical teams may not fully interpret complex model behavior?

I think meaningful transparency requires you to know what it’s getting deployed to. If I go to a different country and unknowingly break the law, I don’t get to say, well, I didn’t know. I think we have to apply the same thing with technology. It’s like if you’re willing to deploy it, then you should be willing to do the due diligence to understand it. And if you don’t understand it, maybe you shouldn’t deploy. 

[Me: Yeah, step away from the big red button. That makes me think of that famous Herbert Hoover quote about engineering]

That’s why we take an obligation, right? It’s a big deal. 

Historically, infrastructure decisions shape institutions for decades. Are companies treating AI architecture decisions with that same level of seriousness?

I would say yes. A lot of the CIOs I deal with understand that these decisions are going to be long-lasting. And they also understand that they’re in a moment where things are moving and changing rapidly. And so there is a lot of cautiousness. 

Five years from now, what assumption about enterprise AI governance do you think companies will most regret making today?

That it will sort itself out. Apathy is always seductive, isn’t it? It’s easy.

For technologists building AI systems: where should responsibility sit between model creators, platforms, and users?

In a perfect world, it should sit at all three levels. The developer should have responsibility over what they develop, the folks implementing it should have responsibility for what they’re implementing, and the user should have responsibility for themselves. So, ideally all three. 

In reality, I think it will most likely land with the implementers, because the technology companies are not going to want to take responsibility for how their tools are used, referring to human culpability instead

Twenty years from now, what will people miss most about the pre-AI internet?

I think we will miss the feeling of the little dopamine hit you got when your crush sent you a message.  

[Me: Omg, Messenger or ICQ! Posting your emo lyrics on there and yes, it was Armageddon when your crush logged on. Nobody understands that particular moment]

Exactly. That’s what I will miss, because now, it’s becoming ubiquitous, right? I just assume everyone is online all the time. We’ve lost a little bit of that human tech interface that was technology-driven, but still directly tied to human reaction emotion, because it wasn’t everywhere. 

[Me: Now I’m doing the Kubrick stare. I didn’t even think about that. I was thinking about social trust, faith in institutions etc. *Sob*] 

Before I let you go, it’s time for a small detour into chaos with Unexpectedly Important Questions. 

Star Wars or Star Trek? 

Star Trek.

[Me: You’re the first one to answer this! Wait, Next Gen?] 

I give the original series a plus one because of all the records they broke. They changed television in ways that Next Gen didn’t. 

[Me: Oh, they were truly groundbreaking, yes]

Mulder or Scully? 

Mulder

Backstreet Boys or N*SYNC?

Neither. Boyz II Men.

[Me: Ohhh, so many hits. They were the soundtrack to millennial dances]

Favourite beverage

Paper plane

Favourite meal 

Oxtail. A nice Jamaican oxtail, rice and peas. 

Thanks Andrew and thanks for reading!

Sponsor this Article