zkML：通过零知识密码学进化智能合约
Proving machine learning (ML) model inference via zkSNARKs is poised to be one of the most significant advancements in smart contracts this decade. This development opens up an excitingly large design space, allowing applications and infrastructure to evolve into more complex and intelligent systems.
By adding ML capabilities, smart contracts can be made more autonomous and dynamic, allowing them to make decisions based on real-time on-chain data, rather than just static rules. Smart contracts would be flexible and adaptable to various scenarios, including those that may not have been anticipated when the contract was initially created. In short, ML capabilities will broaden the automation, accuracy, efficiency, and flexibility of any smart contract we put on-chain.
In many ways, it is surprising that smart contracts do not use embedded ML models today given the prominence of ML in a majority of applications outside of web3. This absence of ML was largely due to the high computational cost of running these models on-chain. For example, FastBERT, a compute-optimized language model, uses ~1800 MFLOPS (million floating point operations), which would be infeasible to run directly on the EVM.
When considering the application of ML models on-chain, the primary focus is with the inference phase: applying the model to make predictions over real-world data. In order to have ML-scaled smart contracts, contracts must be able to ingest such predictions, but as we mentioned earlier, it’s infeasible to run the model directly on the EVM. zkSNARKs give us a solution: anyone can run a model off-chain and generate a succinct and verifiable proof showing that the intended model did in fact produce a particular result. This proof can be published on-chain and ingested by smart contracts to amplify their intelligence.
In this piece, we will:
Machine learning (ML) is a subfield of artificial intelligence (AI) that focuses on developing algorithms and statistical models that enable computers to learn from and make predictions or decisions based on data. ML models typically have three primary components:
Models are produced and deployed in two phases:
zkML today is primarily focused on the inference stage of ML models rather than on the training phase primarily due to the computational complexity of verifying training in-circuit. zkML’s focus on verifying inference is not a limitation though: we expect some very interesting use cases and applications to be produced from the inference phase.
There are four possible scenarios for verified inference:
Verified ML inference opens up a new design space for smart contracts. Some crypto-native applications include:
The ecosystem for zkML can be broadly broken down into four primary categories:
When looking at the zkML ecosystem, most attention has been given to creating model-to-proof compilers. Generally, these compilers translate high-level ML models written in Pytorch, Tensorflow, or the like into zk circuits.
EZKL is a library and command-line tool for doing inference for deep learning models in a zk-SNARK. With EZKL, you can define a computational graph in Pytorch or TensorFlow, export it as a ONNX file with some sample inputs in a JSON file, and point EZKL to these files to generate a zkSNARK circuit. With the latest round of performance improvements, EZKL can now prove an **[MNIST](https://en.wikipedia.org/wiki/MNIST_database#:~:text=The MNIST database (Modified National,the field of machine learning.)**-sized model in about 2 seconds and 700MB of RAM. EZKL has seen some significant early adoption thus far, being used as infrastructure for various hackathon projects to date.
Cathie So’s circomlib-ml library contains various ML circuit templates for Circom. Circuits include some of the most common ML functions. Keras2circom, also built by Cathie, is a python tool that transpiles a Keras model into a Circom circuit using the underlying circomlib-ml library.
LinearA has developed two frameworks for zkML: Tachikoma and Uchikoma. Tachikoma is used to convert neural networks into an integer-only form and generate a computational trace. Uchikoma is a tool that converts TVM’s intermediate representation into programming languages that don’t support floating point operations. LinearA plans to support Circom, which uses field arithmetic, and Solidity, which uses signed and unsigned integer arithmetic.
Daniel Kang’s zkml is a framework for constructing proofs of ML model execution in ZK-SNARKs based on his work in the Scaling up Trustless DNN Inference with Zero-Knowledge Proofs paper. As of the time of this writing, it is capable of proving an MNIST circuit using around 5GB of memory and around 16 seconds to run.
On the more generalized model-to-proof compilers, there is both Nil Foundation and Risc Zero. Nil Foundation’s zkLLVM is an LLVM-based circuit compiler and is capable of verifying computational models written in popular programming languages such as C++, Rust, and JavaScript/TypeScript, among others. It is generalized infrastructure versus some of the other model-to-proof compilers noted here, but it is still suitable for complex computations like zkML. This can be particularly powerful when combined with their proof market.
Risc Zero has built a general-purpose zkVM tagetting theopen-source RISC-V instruction set, hence supporting existing mature languages such as C++ and Rust, as well as the LLVM toolchain. This allows for seamless integration between host and guest zkVM code, similar to Nvidia’s CUDA C++ toolchain, but with a ZKP engine in place of a GPU. Similar to Nil, it is possible to verify the computational trace of an ML model using Risc Zero.
Improvements in proving systems have been the primary enabler in bringing zkML to fruition, specifically the introduction of custom gates and lookup tables. This is primarily due to ML’s reliance on nonlinearities. In short, nonlinearities are introduced through activation functions (e.g. ReLU, sigmoid, and tanh), which are applied to the outputs of linear transformations within a neural network. These nonlinearities are challenging to implement in zk circuits due to limitations around mathematical operation gates. Bitwise decomposition and lookup tables can help solve this problem by precomputing the possible outcomes of a nonlinearity into a lookup table, which interestingly is much more computationally efficient in zk.
**[Plonkish](https://zcash.github.io/halo2/concepts/arithmetization.html#:~:text=PLONKish circuits are defined in,will be elements of F.)** proof systems tend to be the most popular backends for zkML for this reason. Halo2 and Plonky2 with their table-style arithmetization scheme can handle neural network non-linearities well via lookup arguments. In addition, the former has a vibrant developer tooling ecosystem coupled with flexibility, making it the de facto backend for many projects including EZKL.
Other proof systems have their benefits as well. R1CS-based proof systems include Groth16 for its small proof sizes and Gemini for its handling of extremely large circuits and linear time prover. STARK-based systems like the Winterfell prover/verifier library are also useful especially when implemented via Giza’s tooling that takes a Cairo program’s trace as an input and generates a STARK proof using Winterfell to attest to the correctness of the output.
Some progress has been made in designing efficient proof systems that can handle the complex, circuit-unfriendly operations of advanced ML models. Systems like zkCNN, which is based on the GKR proving system, or the use of compositional techniques in the case of systems like Zator are often more performant than their generalized counterparts as evidenced by the Modulus Labs’ benchmarking report.
zkCNN is a method for proving the correctness of convolutional neural networks using zero knowledge proofs. It uses a **sumcheck protocol**to prove fast Fourier transforms and convolutions with a linear prover time that is faster than computing the result asymptotically. Several improvements and generalizations have been introduced for interactive proofs, including verifying the convolutional layer, the ReLU activation function, and max pooling. zkCNN is particularly interesting given Modulus Labs’ report on benchmarking where they found it outperformed other generalized proving systems in both proof generation speed and RAM consumption.
Zator is a project aimed at exploring the use of recursive SNARKs to verify deep neural networks. The current constraint for verifying deeper models is fitting the entire computation trace into a single circuit. Zator proposes verifying one layer at a time using recursive SNARKs, which can verify N-step repeated computation incrementally. They use Nova to reduce N instances of computation into a single instance that can be verified at the cost of a single step. With this approach, Zator was able to snark a network with 512 layers, which is as deep as most production AI models today. Proof generation and verifying time for Zator are still too large for mainstream use cases, but their compositional technique is interesting nonetheless.
Much of the focus of zkML given its early stage has been on the above infrastructure. However, there are a few projects working on applications today.
Modulus Labs is one of the most diverse projects within the zkML space, working on both example applications and relevant research. On the application side, Modulus Labs has demonstrated zkML use cases through RockyBot – an on-chain trading bot – and Leela vs. the World – a chess game where all of humanity plays against a verified, on-chain instance of the Leela chess engine. The team has also branched into research, writing The Cost of Intelligence which benchmarked various proving systems for the speed and efficiency across differing model sizes.
Worldcoin is applying zkML in its attempt to make a privacy-preserving proof of personhood protocol. Worldcoin is using custom hardware to process high-resolution iris scans which are inserted into their Semaphore implementation. This can then be used to perform useful operations like membership attestation and voting. They currently use a trusted runtime environment with a secure enclave to verify the camera-signed iris scan, but they ultimately aim to use a ZKP to attest to the correct inference of the neural network for cryptographic-level security guarantees.
Giza is a protocol that enables the deployment of AI models on-chain using a fully trustless approach. It uses a technology stack that includes the ONNX format for representing machine learning models, the Giza Transpiler for converting these models into the Cairo program format, the ONNX Cairo Runtime for executing the models in a verifiable and deterministic way, and the Giza Model smart contract for deploying and executing the models on-chain. While Giza could also fit into the Model-to-Proof compiler category, their positioning as a marketplace for ML models is one of the more interesting applications out there today.
Gensyn is a distributed hardware provisioning network used to train ML models. Specifically, they are engineering a probabilistic audit system based on gradient descent and using model checkpoints to enable a decentralized GPU network to service the training of full-scale models. While their zkML application here is highly specific to their use case – they want to ensure that when a node downloads and trains a piece of a model that they are honest about the model updates – it showcases the power of combining zk and ML.
ZKaptcha is focused on the bot problem within web3, providing captcha services for smart contracts. Their current implementation has the end user essentially producing a proof of human work by completing the captcha, which is verified by their on-chain verifier and accessible by a smart contract via a couple lines of code. Today, they are primarily only relying on zk, but they intend to implement zkML in the future similar to existing web2 captcha services that analyze behaviors like mouse movements to determine if a user is human.
Given how early the zkML market is, a lot of applications have been experimented with at the hackathon level. Projects include AI Coliseum, an on-chain AI competition built using EZKL that uses ZK proofs to validate machine learning outputs, Hunter z Hunter, a photo scavenger hunt using the EZKL library to validate an image classification model’s outputs with halo2 circuits, and zk Section 9, which converted an AI image generating model into a circuit for minting and verification of AI art.
While improvements and optimizations are being made at light-speed, some core challenges remain for the zkML space. These challenges range from technical to practical and include:
Quantization is the process of representing floating-point numbers, which most ML models use to represent model parameters and activations, as fixed-point numbers, which is necessary when dealing with the field arithmetic of zk circuits. The impact of quantization on accuracy for machine learning models depends on the level of precision that is used. Generally, using lower precision (i.e., fewer bits) can result in reduced accuracy because it can introduce rounding and approximation errors. However, there are several techniques that can be used to minimize the impact of quantization on accuracy, such as fine-tuning the model after quantization and using techniques like quantization-aware training. In addition, Zero Gravity – a hackathon project at zkSummit 9 – has shown that alternative neural network architectures developed for edge devices such as the weightless neural network can be useful for avoiding the problems of quantization in-circuit.
Outside of quantization, hardware is another key challenge. Once a machine learning model is properly represented via a circuit, it is cheap and fast to verify proofs of its inferences due to the succinctness property of zk. The challenge here is not on the verifier, but on the prover since RAM consumption and proof generation time scales quickly as the model size grows. Certain proving systems (e.g. GKR-based systems that use sumcheck protocols and layered arithmetic circuits) or compositional techniques (e.g. wrapping Plonky2, which is efficient at proving time but poor at efficient proof sizes for large models, with Groth16, which doesn’t see a growing proof size with complexity of the model) are better suited for handling these issues, but managing tradeoffs is a core challenge for projects building in zkML.
On the adversarial side, there is also still work to be done. For one, if a trustless protocol or DAO elects to implement a model, there is still risk around adversarial attacks during the training phase (e.g. training a model to behave a specific way when it sees an input which could be used to manipulate later inferences). Federated learning techniques and training phase zkML could be one way of minimizing this attack surface.
Another core challenge is the risk of model-stealing attacks when a model is privacy-preserved. While a model’s weights can be obfuscated, it is theoretically possible to reverse engineer the weights given enough input-output pairs. This is mainly a risk for smaller scale models, but it is a risk nonetheless.
Although there have been some challenges with optimizing these models to operate within the constraints of zk, improvements are being made at exponential speed and some anticipate we will reach parity with the broader ML space soon assuming further hardware acceleration. To underscore the pace of these improvements, zkML has gone from 0xPARC’s zk-MNIST demonstration in 2021 showing how a small-scale **[MNIST](https://en.wikipedia.org/wiki/MNIST_database#:~:text=The MNIST database (Modified National,the field of machine learning.)** image classification model could be performed in a verifiable circuit to Daniel Kang’s paper doing the same for ImageNet-scale models just under a year later. In April of 2022, this ImageNet-scale model was further improved from 79% accuracy to 92% accuracy, and networks as large as **GPT-2**are potentially feasible in the near-term albeit with slow proving times today.
We view zkML as a rich and growing ecosystem that looks to scale the capabilities of blockchains and smart contracts to be more flexible, adaptable, and intelligent.
While zkML is still in its early stages of development, it has already begun to show promising results. As the technology evolves and matures, we can expect to see even more innovative use cases for zkML on-chain.
https://en.foresightnews.pro/zkml-evolving-the-intelligence-of-smart-contracts-through-zero-knowledge-cryptography/
通过 zkSNARKs 证明机器学习 (ML) 模型推理有望成为这十年智能合约最重要的进步之一。 这一发展开辟了一个激动人心的大设计空间，允许应用程序和基础设施发展成为更复杂和智能的系统。
通过添加 ML 功能，智能合约可以变得更加自主和动态，能够基于实时链上数据做出决策，而不仅仅是静态规则。智能合约将变得灵活和适应各种情况，包括在合约最初创建时可能没有预料到的情况。简而言之，机器学习能力将扩大我们放在链上的任何智能合约的自动化、准确性、效率和灵活性。
在许多方面，令人惊讶的是，智能合约在当今并没有使用嵌入式的机器学习模型，尽管机器学习在Web3以外的大多数应用中都非常突出。这种缺乏机器学习的原因主要是由于在链上运行这些模型的计算成本很高。例如，一个经过计算优化的语言模型FastBERT需要大约1800 MFLOPS（百万次浮点运算），这在以太坊虚拟机（EVM）上直接运行是不可行的。
在考虑链上 ML 模型的应用时，主要关注的是推理阶段：应用模型对真实世界的数据进行预测。为了拥有 ML 规模的智能合约，合约必须能够摄取此类预测，但正如我们之前提到的，直接在 EVM 上运行模型是不可行的。zkSNARKs 为我们提供了一个解决方案：任何人都可以在链下运行一个模型，并生成一个简洁且可验证的证明，证明预期的模型确实产生了特定的结果。这个证明可以发布到链上，并被智能合约接受，以增强它们的智能性。
在本文中，我们将会：
机器学习 (ML) 是人工智能 (AI) 的一个子领域，专注于开发算法和统计模型，使计算机能够从数据中学习并根据数据做出预测或决策。ML 模型通常由三个主要组成部分构成：
模型的生成和部署分两个阶段进行：
目前，zkML主要关注 ML 模型的推理阶段，而不是训练阶段，主要是因为在电路中验证训练的计算复杂性。然而，zkML对验证推理的关注并不是一种限制：我们预计会有一些非常有趣的用例和应用从推理阶段产生。
有四种可能的经过验证的推理场景：
私有输入，公共模型。
模型使用者（MC）可能希望将他们的输入对模型提供者（MP）保密。例如，MC可能希望向贷款人证明信用评分模型的结果，而不需披露个人财务信息。这可以通过使用预承诺方案并在本地运行模型来实现。
公共输入，私有模型。ML-as-a-Service
的一个常见问题 是 MP 可能希望隐藏其参数或权重以保护其 IP，而 MC 想要验证生成的推理确实来自对抗设置中的指定模型. 这样想：当向 MC 提供推理时，MP 有动机运行更轻的模型以节省成本。使用链上模型权重的承诺，MC 可以随时审计私有模型。
私有输入，私有模型。
当用于推理的数据高度敏感或机密，并且为了保护知识产权而隐藏了模型时，就会出现这种情况。一个例子是使用私人患者信息审核医疗模型。可以使用zk中的组合技术或多方计算（MPC）或FHE的变体来处理此场景。
公共输入，公共模型。
当模型的所有方面都可以公开时，zkML 解决了一个不同的用例：将链下计算压缩和验证转移到链上环境中。对于较大的模型，验证推理的简洁 zk 证明比重新运行模型本身更具成本效益。
Verified ML 推理为智能合约开辟了一个新的设计空间。一些加密原生应用程序包括：