Trustworthy AI frameworks in Asia — analysing China and Singapore’s published guidance for Generative AI
This article represents a personal opinion only.
Many Generative AI governance frameworks have been published, with impliciations for a technology practitioners day-to-day work. This article distills the common areas of focus between these frameworks, and comments on what this means for the people, processes and technologies within the technology industry.
An overview of the field globally
In October 2023, US president Joe Biden signed an executive order that outlined 8 principles and priorities for “the safe, secure, trustworthy development of Artificial Intelligence”. Similarly, the UK also released a whitepaper proposing how the government should regulate Artificial Intelligence (AI).
Closer to home, Singapore also released a “Proposed Model Governance Framework for Generative AI”, while China published their own “Measures for the Management of Generative AI services”.
It’s only natural that many frameworks, from different countries, are being published rapidly. AI is a transformative technology that governments feel compelled to control and regulate. Indeed, the authors of all the frameworks acknowledge Generative AI’s “potential for both promise and peril”.
In their papers, the authors recognise the immense potential of AI to discover new medicines, improve public services and remove manual toil from workers. At the same time, they also acknowledge how Generative AI, trained as it is on data that unavoidably reflects societal biases, can also perpetuate existing inequalities and entrench misinformation.
As the author of The Pragmatic Engineer, Gergery Orosz phrases it, “in doubt that regulation would arrive for such an innovative, disruptive field as AI. The only question is how quickly it would happen”.
What this means for practitioners
As technologists, how we work and how we build tools for our users will be affected by these guidelines. Already, procurement departments who publish Request For Information questionnaires when onboarding a new platform or tool are asking about how an AI service can meet the fairness, anti-bias and data privacy requirements these frameworks ask for. Because of the increased need for data classification and lineage, data engineers will need to build in validation, tagging and monitoring into their pipelines more than ever.
To help you understand these incoming changes, this article distills the common areas of focus between these frameworks, with a special emphasis on the guidelines published by Singapore and China. We’ll see that there are many areas common to both frameworks. They all mention the importance of respecting an individual’s data rights, call on the need for more transparent and rigorously tested models, and draw out the need to build user awareness of how Generative AI will affect them.
Mostly though, these guidelines are, by themselves, too general to be effective. Most likely, it is an organization’s responsible parties, like Data Governance officers, who will have to translate these guidelines into specific practices around how a company manages its data and develops its models. This is something we will explore in future posts. We will examine how a company might implement these guidelines, from the data layer all the way to the application layer.
Here are some key takeaways:
Emphasis on preserving the rights on an individual
- Right to data privacy and fair model outputs
Mentioned in both frameworks are how data, whether used to train a model or to get a model’s prediction, should be used in accordance with existing data privacy practices. A model’s output should also, as far as possible, not should not discriminate against certain classes of users.
In practice, this puts focus on data governance tools that typically manage data access controls, specify data privacy features and provide audit logging. Data governance is a large field in itself; suffice to say that Generative AI governance will only make the field more important.
Stress on the need for more rigorously evaluated and transparent models
Both frameworks emphasised the need for more control over the output of Generative AI models (through evaluation and testing) and more visibility into a models provenance (through tracking the model’s training data and model’s risks and limitations). In a way, this second point is related to the one on an individual’s data rights: only through rigorous evaluation and visibility can one guarantee that user rights are respected.
- Rigorous evaluation
From an engineering point of view, there is a definite imbalance in this area. The tools for creating evaluation pipelines are more mature than the actual tests for evaluating a model’s robustness (for example how factual a model is or aligned it is to a society’s moral values). Existing CI/CD pipelines can integrate model evaluations by calling APIs such as MLflow LLM Evaluate API. Before deployment, existing tools such as Azure DevOps allow for manual approvals in order for a CI/CD pipeline to succeed.
There are, however, few agreed-upon tests that measure a model’s factualness, tone, and, most importantly for users, relevancy. There are some frameworks, for example RAGAS and FAVA, and of course the good old standby — having a human-in-the-loop- but a commonly-agreed upon metric does not exist.
In fact, this may even be the status quo for a long time. Generative AI can product different phrasings of the same answer, or answer a question equally factfully using different sources. In these subjective cases, definitive evaluation is hard.
Here, the amount of effort put into evaluating a model will come down to how “material” a model is, or the extent to which a model’s outputs affects a user. A model who determines financial products to recommend to a user will have a higher materiality than a model that converts a user’s profile picture into an avatar. Hence, the recommender model will need more scrutiny.
- Transparency
Within this context, transparency refers to disclosing clearly how the model was made and what data was used to make it. The Singapore Model Goverance Framework gives a useful analogy here — “food labels”. For a certain food product, such as mayonnaise, food labels contain a list of the ingredients that make up a food product. A food label may also contain information on where the product was made and packaged and who its distributor is. Similarly, a Generative AI model should have information on the data used to train it, the model’s intended use, and its risks and limitations (source). Consumers can then be more informed about the models that power the tools they use, from email to calendars to cars.
In practice, transparency is cannot be separated from good lineage tracking. Lineage tracking, in itself, can be broken down into two components. It can refer to tracking a model’s training data and inputs at inference time. It can also mean having visibility into the sub-components of a model. Implementing the former means having good dataset tagging practices, informative metadata and clear dataset descriptions. For the latter, tracing the flow of information through each of the model’s sub-components becomes important.
There are various ways model tracing might happen, but here is one example — to understand why a model is giving less than factual answers, a developer might comparing the relative accuracy of the retrieval step (where a model searches through documents for relevant information) versus the generation step (where a model uses relevant documents to generate a user response).
Tracing is something that has been extensively studied in other software engineering disciplines such as API observability. Some of the lessons from this disciplines, such as implementing trace graphs, have already made their way into open-source Large Language Model tracing tools such as Langfuse, to promising results.
Adding good metadata to dataset, however, can be harder to tackle. Adding metadata like table descriptions has traditionally relied on manual effort, meaning that many tables in an organisation can go undocumented. A promising approach is to use Large Language models to automatically generate table descriptions, as Databricks has done.
Addresses the need for building user awareness
- flag content created by Generative AI
User awareness is another theme emphasised by both Generative AI governance frameworks. Both frameworks encourage the use of “watermarks” to delineate content that is generated by AI, and more broadly to inform consumers on the authenticity of the content they see digitally.
Flagging content as “AI-generated” goes some way towards addressing problems of misinformation. However, user awareness is a broader field than just preventing information misuse. It also includes other considerations, considerations such as helping users to more savvy about the technology they use. For example, users should made aware that recommendations they are seeing can lead to “filter bubbles”, where content they bookmark leads them to be recommended similar content. Users should also know that can control whether their timelines are algorithm-driven or simply timeline driven.
In this area, the biggest wins here will come from thoughtful user design. Useful frameworks have already been published for this, for instance Google’s People + AI Guidebook. Design patterns, case studies and workshops will make a difference here.
Conclusion
Much of what these governance frameworks emphasise might seem self-evident. Principles like data privacy, detailed provenance and user awareness are features that every application should have. Yet in a regulatory landscape that’s still fragmented and susceptible to the third-party interests, reiterating these inviolable principles can be a form of reassurance. It’s also a reminder that because Generative AI technology affects how we make decisions and how we live, human values remain at its centre.
