零知识机器学习 (ZKML) 简介
https://worldcoin.org/blog/engineering/intro-to-zkml
A zero-knowledge (ZK) proof is a cryptographic protocol in which one party, the prover, can prove to another party, the verifier, that a given statement is true, without revealing any additional information beyond the fact that the statement is true. It is an area of study that has been making great progress on several fronts, from research to protocol implementations and applications.
The two main “primitives” (or, building blocks) that ZK brings to the table are the ability to create proofs of computational integrity for a set of given computations, where the proof is significantly easier to verify than it is to perform the computation itself. (We call this property “succinctness.”) ZK proofs also provide the option to hide parts of said computation whilst preserving computational correctness. (We call this property “zero-knowledge.”)
Generating zero-knowledge proofs is very computationally intensive, many times as expensive as the original computation. This means that there are some computations for which it is infeasible to compute zero-knowledge proofs because the time it'd take to create them on the best hardware available makes them impractical. However, advancements in the field of cryptography, hardware, and distributed systems in recent years have allowed zero-knowledge proofs to become feasible for ever more intensive computations. These advancements have allowed for the creation of protocols that can use proofs of intensive computations, thus expanding the design space for new applications.
Zero-knowledge cryptography is one of the most popular technologies in the Web3 space since it allows developers to build scalable and/or private applications.
As ZK tech matures it's likely there will be a Cambrian explosion of new applications since the tooling used to build them will require less domain expertise and will be a lot easier to use for developers.
Below are a few examples of how it is being used in practice (though note that many of these projects are works-in-progress).
Distributed systems like public blockchains have limited computational power since all participant nodes (computers) have to verify the computations in each block by running them themselves. Using zk proofs we can execute these computations off-chain, compute a zk proof and verify this proof on-chain, thus achieving scalability without sacrificing decentralization or security. Examples:
The zero-knowledge property of ZK proofs enables hiding parts of the computation being proven which is really useful to create applications that preserve users' privacy and personal data when making cryptographic attestations. Examples:
Aztec is building a private scalability solution for Ethereum (ZK rollup) where users' balances and transactions are fully hidden to any outside observer.
At Worldcoin we are building WorldID, which is our privacy-preserving proof-of-personhood protocol. It allows any person with a WorldID to make a cryptographic attestation signalling that they are a unique human being and that they haven't performed an action before (like signing up for a social network) without revealing their identity.
Since ZK proofs help off-load computation and make computations private it allows for the creation of private and/or succinct (small in size, easily verifiable) layer 1s. Examples:
Machine learning is a subfield of artificial intelligence that involves the development and application of algorithms that enable computers to learn and adapt from data autonomously, optimizing their performance through iterative processes. Large language models, such as GPT-4 and Bard, are state-of-the-art natural language processing systems that leverage vast amounts of training data to generate human-like text, while text-to-image models like DALL-E 2, Midjourney, and Stable Diffusion translate textual descriptions into visual representations with remarkable fidelity. The rapid advancement of machine learning techniques holds significant promise in addressing complex challenges across various domains, including healthcare, finance, and transportation, by leveraging data-driven insights and predictions to improve decision-making and optimize outcomes. As these models become more sophisticated, they are poised to revolutionize numerous industries, transforming the way we live, work, and interact with technology.
In a world where AI-generated content increasingly resembles human-created content, the potential application of zero-knowledge cryptography could help us determine that a particular piece of content was produced by applying a specific model to a given input. This could provide a means for verifying outputs from large language models like GPT4, text-to-image models such as DALL-E 2, or any other models, if a zero-knowledge circuit representation is created for them. The zero-knowledge property of these proofs would allow us to also hide parts of the input or the model as well if need be. A good example of this would be applying a machine learning model on some sensitive data where a user would be able to know the result of model inference on their data without revealing their input to any third party (e.g., in the medical industry).
Note: When we talk about ZKML, we are talking about creating zero-knowledge proofs of the inference step of the ML model, not about the ML model training (which, in and of itself, is already very computationally intensive).
The current state of the art of zero-knowledge systems coupled with performant hardware still falls a few orders of magnitude short of being able to prove something as big as currently available large language models (“LLMs”), but there has been some progress in creating proofs of smaller models.
We researched the state of the art of zero-knowledge cryptography in the context of creating proofs for ML models and created an aggregation of the relevant research, articles, applications, and codebases that belong to this domain. Resources on ZKML can be found on the ZKML community's awesome-zkml repository on GitHub.
The Modulus Labs team recently released a paper titled “The Cost of Intelligence”, where they benchmark existing ZK proof systems against a wide range of models of different sizes. It is currently possible to create proofs for models of around 18M parameters in about 50 seconds running on a powerful AWS machine using a proving system like plonky2. Figure 1 illustrates the scaling behavior of different proving systems as the number of parameters of a neural network are increased:
Fig. 1Source: “The Cost of Intelligence: Proving Machine Learning Inference with Zero-Knowledge.” Modulus Labs. Fig. 2, pp. 12. January 20, 2023.
Another initiative that is working on improving the state of the art of ZKML systems is Zkonduit's ezkl library which allows you to create ZK proofs of ML models exported using ONNX. This enables any ML engineer to create ZK proofs of the inference step of their models and to prove the output to any verifier.
There are several teams working on improving ZK technology, creating optimized hardware to speed up the computation of ZK proofs, particularly for resource-intensive tasks such as the prover and verifier algorithms. As ZK technology matures it will be possible to prove bigger models on less powerful machines in a smaller period of time due to improvements in specialized hardware, proof system architecture (proof size, verification time, proof generation time, etc) and more performant ZK protocol implementations. We expect these advancements will allow new ZKML applications and use cases to emerge.
In order to decide whether ZKML could be used for a given application, we can examine how the properties of ZK cryptography would help enable certain use cases. This can be illustrated as a Venn Diagram:
Fig. 2Venn Diagram explaining how ZK and ML primitives and technologies can be combined together
Heuristic optimization - A problem-solving approach that uses rules of thumb or "heuristics" to find good solutions to problems that are difficult to solve using traditional optimization methods. Rather than trying to find the optimal solution to a problem, heuristic optimization methods aim to find a good or "good enough" solution in a reasonable amount of time given the relative importance of the problem to the overall system and the difficulty in optimizing it.
Fully Homomorphic Encryption (FHE) ML - FHE allows developers to perform operations on encrypted data and when decrypted the result will be the output of the operation performed on the original unencrypted input. Enables evaluating models in a privacy-preserving fashion (full data privacy, unlike ZKML where the prover needs access to all data); however, there's no way to cryptographically prove the correctness of the computations being performed like with ZK proofs. For example, Zama is working on creating a FHE ML framework called Concrete ML.
ZK proofs vs. Validity proofs - These terms are oftentimes used interchangeably in the industry since validity proofs are ZK proofs that don't hide parts of the computation or its results. In the context of ZKML, most current applications are leveraging the validity proof aspect of ZK proofs.
Validity ML - [SNARK/STARK](https://blog.chain.link/zk-snarks-vs-zk-starks/#:~:text=SNARKs and STARKs are zero,without revealing any further information.) proofs of ML models where all computations are publicly visible to the verifier (). Any verifier can then prove the computational correctness of the ML models.
ZKML - ZK proofs of ML models where computations are being hidden from the verifier (using the zero-knowledge property). The prover can prove the computational correctness of the ML models without revealing any further information.
Validity proofs (SNARKs/STARKs) can be used to prove that some computation happened correctly, in the context of ML we are proving ML model inference or that some model created some output using a specific input.
For example Modulus Labs, a ZKML-focused startup, is building these use cases:
The ability to easily prove and verify that the output is the product of a given model and input pair. This enables ML models to be run off-chain on specialized hardware and to have their ZK proofs easily verifiable on-chain. For example, Giza is helping Yearn (a DeFi yield aggregator protocol) to prove that some complex yield strategy which is using ML is being correctly executed on-chain.
When different companies provide access to ML models through their APIs, it is really hard to know as a user whether the service provider is actually providing the model that they say they are since the API is a black box. Providing validity proofs attached to an ML model API would be useful to provide transparency to the user as they can verify which model they are using.
Enables the creation of a ZK proof for exploitability/fraud. Anomaly detection models could be trained on smart contract data and agreed upon by DAOs as interesting metrics to be able to automate security procedures such as pausing contracts in a more proactive, preventive way. There are startups already looking at using ML models for security purposes in a smart contract context, so ZK anomaly detection proofs could be a next step.
Besides validity proofs, we can also hide parts of the computation in order to enable the privacy-preserving application of ML. A few examples can be found below:
One potential use of ZKML in the context of Worldcoin is iris code upgradeability. World ID users would be able to self-custody their signed biometrics in the encrypted storage of their mobile device, download the ML model for iris code generation and create a zero-knowledge proof locally that proves their iris code was indeed generated from signed images using the correct model. This iris code could then be permissionlessly inserted into the set of registered Worldcoin users since the receiving smart contract would be able to verify the zero-knowledge proof which validates the creation of the iris code. This would mean that, if Worldcoin ever upgrades the algorithm to create the iris code in a way that'd break compatibility with its previous iteration, users wouldn't have to go back to an Orb, but could just compute the upgrade locally on-device.
零知识（ZK）证明是一种密码协议，其中一方（证明者），可以向另一方（验证者）证明给定的语句是真实的，而无需透露除该陈述为真之外的任何其他信息。这是一个正在多个方面取得巨大进展的研究领域，从研究到协议实现和应用。
ZK带来的两个主要“原语”（或构建块）是能够为一组给定的计算创建计算完整性证明，其中证明在验证时比执行计算本身要容易得多。 （我们称这种属性为“简洁性。”）ZK证明还提供了隐藏所述计算的部分并保持计算正确性的选项。 （我们称这种属性为“零知识。”）
生成零知识证明可能需要很大的计算量，比原始计算本身更昂贵。某些计算可能无法计算零知识证明，因为在现有硬件上创建它们所需的时间使其变得不切实际。然而，近年来密码学，硬件和分布式系统领域的进步使得零知识证明在越来越强大的计算中变得可行。这些进展扩大了零知识协议的潜在应用和设计空间。
零知识密码学是Web3领域中最流行的技术之一，因为它允许开发人员构建可扩展和/或私有应用程序。
随着ZK技术的成熟，可能会出现大量新应用程序的巨大爆发，因为用于构建这些应用的工具将需要更少的领域专业知识，并且对于开发人员来说将更容易使用。
以下是一些实际应用的示例（但请注意，其中许多项目仍在进行中）。
像公共区块链这样的分布式系统由于所有参与节点（计算机）都必须自己运行每个块中的计算来验证计算，因此具有有限的计算能力。使用zk证明，我们可以在链下执行这些计算，计算zk证明并在链上验证此证明，从而实现可扩展性而不牺牲去中心化或安全性。例如：
ZK证明的零知识特性使得隐藏被证明的计算的部分成为可能，这对于创建保护用户隐私和个人数据的应用程序非常有用，尤其是在进行加密证明时。例如：
Aztec正在为以太坊构建一个私有的可扩展性解决方案（ZK rollup），其中用户的余额和交易对任何外部观察者完全隐藏。
在Worldcoin中，我们正在构建WorldID，这是我们的保护隐私的人类身份证明协议。它允许任何拥有WorldID的人进行加密证明，表明他们是独一无二的人类，并且他们以前没有执行过某个操作（比如注册社交网络），而不暴露他们的身份。
由于ZK证明可以卸载计算并使计算保密，因此可以创建私有和/或简洁（大小小，易于验证）的第1层协议。例如：
机器学习是人工智能的一个子领域，涉及开发和应用算法，使计算机能够自主地从数据中学习和适应，通过迭代过程优化其性能。大型语言模型（例如GPT-4和Bard）是最先进的自然语言处理系统，利用大量的训练数据生成类似于人类的文本，而文本到图像模型（例如DALL-E 2，Midjourney和Stable Diffusion）则将文本描述转换为具有显著保真度的视觉表示。机器学习技术的快速发展在各个领域都具有重要的应用前景，包括医疗保健、金融和交通等，通过利用数据驱动的洞见和预测来改善决策和优化结果。随着这些模型变得越来越复杂，它们有望彻底改变许多行业，改变我们生活、工作和与技术互动的方式。