Analyzing the potential and practical challenges of combining AI and encryption

Author: @ed_roman; Translator: Plain Language Blockchain

Recently, artificial intelligence has become one of the hottest and most promising areas in the crypto market. Including:

• Decentralized AI training

• GPU decentralized physical infrastructure network

• Uncensored AI models Are these groundbreaking developments or just hype?

At @hack_vc, we are working hard to clear the fog and separate promises from reality. This article will deeply analyze the top ideas in crypto and AI. Let's explore the real challenges and opportunities together.

1. Challenges of combining Web3 with AI

1. Decentralized AI training

The problem with AI training on the chain is that training requires high-speed communication and coordination between GPUs, because neural networks need to be back-propagated during training. Nvidia provides two innovative technologies for this purpose (NVLink and InfiniBand). These technologies can greatly speed up GPU communication, but they can only be used in GPU clusters within a single data center (speeds exceeding 50 Gbps).

If a decentralized network is introduced, the speed will be significantly slower due to increased network latency and bandwidth. This is not feasible for AI training use cases at all compared to the high-speed interconnection provided by Nvidia in data centers. In addition, the network bandwidth and storage costs in a decentralized environment are also much higher than those of solid-state drives in local clusters.

Another problem with training AI models on the chain is that this market is less attractive than inference. Currently, a lot of GPU computing resources are used for training AI large language models (LLMs). But in the long run, inference will become the main application scenario for GPUs. Think about it: how many AI large language models need to be trained to meet demand? In contrast, how many customers will use these models?

Note that there are some innovations in this area that may provide hope for the future of on-chain AI training:

1) Distributed training based on InfiniBand is being carried out at a large scale, and NVIDIA itself supports non-local distributed training through its collective communication library. However, this is still in the early stages and adoption remains to be seen. The bottleneck caused by physical distance still exists, so local InfiniBand training is still significantly faster.

2) Some new research has been published that explores decentralized training with reduced communication synchronization times, which may make decentralized training more practical in the future.

3) Smart sharding and training scheduling can help improve performance. Similarly, in the future there may be new model architectures designed specifically for distributed infrastructure (Gensyn is conducting research in these areas).

4) Innovations such as Neuromesh attempt to achieve distributed training at a lower cost through a new approach called predictive coding networks (PCNs).

2. Decentralized AI data iteration

The data information part of training is also a difficult problem. Any AI training process involves processing a large amount of data. Typically, models are trained on centralized and secure data storage systems that are highly scalable and high-performance. This requires transferring and processing terabytes of data, and this is not a one-time cycle. Data is often noisy and contains errors, so before training the model, the data must be cleaned and transformed into a usable format. This stage involves repetitive tasks of standardization, filtering, and handling missing values. In a decentralized environment, these pose serious challenges.

The data information part of training is also iterative, which is not very compatible with Web3. It took OpenAI thousands of iterations to achieve their results. The training process is iterative: if the current model does not achieve the expected results, experts will return to the data collection or model training stage to improve the results. Now, imagine doing this process in a decentralized environment, and the best existing frameworks and tools are not easily available in Web3.

One ​​promising technology is 0g.ai (backed by Hack VC), which provides on-chain data storage and data availability infrastructure. They have a faster architecture and the ability to store large amounts of data on-chain.

3. Reach consensus with overly redundant AI reasoning calculations

One ​​challenge of combining encryption with AI is to verify the accuracy of AI reasoning, because you cannot fully trust a single centralized party to perform reasoning operations, and there is the possibility of node misbehavior. In Web2 AI, this challenge does not exist because there is no decentralized consensus system.

One ​​solution is redundant computing, where multiple nodes repeat the same AI inference operation in order to operate in a trustless environment and avoid single points of failure.

The problem with this approach is that we live in a world where there is a severe shortage of high-end AI chips. The waiting period for high-end NVIDIA chips is as long as several years, which leads to rising prices. If you also require AI inference to be repeated multiple times on multiple nodes, this will significantly increase these expensive costs. For many projects, this is not feasible.

4. Web3-specific AI use cases (in the short term)

Some people suggest that Web3 should have its own unique AI use cases, specifically for Web3 customers.

At present, this is still an emerging market and use cases are still being discovered. Some challenges include:

• Web3 native use cases require less AI transaction volume because the market demand is still in its infancy.

• There are fewer customers because Web3 customers are orders of magnitude smaller than Web2 customers, so the market is less fragmented.

• The customers themselves are less stable because they are startups with less funding, so these startups may fail over time. AI service providers targeting Web3 customers may need to re-acquire some customers over time to replace those that fail, making it more difficult to scale their business.

In the long term, we are very bullish on Web3 native AI use cases, especially as AI agents become more prevalent. We envision a future where every Web3 user has multiple AI agents helping them. An early leader in this space is Theoriq.ai, which is building a platform for composable AI agents that can serve Web2 and Web3 clients (backed by Hack VC).

5. Decentralized Physical Infrastructure Network (DePIN) with Consumer-Grade GPUs

There are many decentralized AI computing networks that rely on consumer-grade GPUs rather than GPUs in data centers. Consumer-grade GPUs are suitable for low-end AI reasoning tasks or consumer use cases with flexible latency, throughput, and reliability requirements. But for serious enterprise use cases (i.e., use cases that occupy a major market share), customers want networks that are more reliable than home machines, and complex reasoning tasks typically require higher-end GPUs. For these more valuable customer use cases, data centers are more suitable.

It should be noted that we believe that consumer-grade GPUs are suitable for demonstration purposes or individuals and startups who can tolerate lower reliability. But the value of these customers is generally lower, so we believe that decentralized physical infrastructure networks (DePIN) for Web2 enterprises will be more valuable in the long run. As a result, well-known GPU DePIN projects have generally evolved from using primarily consumer-grade hardware in the early days to now having A100/H100 and cluster-level availability.

Second, practical and feasible use cases for encryption x AI

Now, let's discuss the use cases where encryption x AI can significantly increase value.

Real benefit 1: Serving Web2 customers

McKinsey estimates that generative AI can bring $2.6 trillion to $4.4 trillion in added value each year for the 63 use cases they analyzed - by comparison, the UK's total GDP in 2021 was $3.1 trillion. This would increase the impact of all AI by 15% to 40%. If we embed generative AI into software currently used for other tasks, the value of this estimate will roughly double.

Funny thing:

• Based on the above estimates, this means that the total market value of global AI (not just generative AI) could be in the tens of trillions of dollars.

• In comparison, the total value of all cryptocurrencies (including Bitcoin and all altcoins) combined is only about $2.7 trillion today.

So, let’s be realistic: the vast majority of customers who need AI in the short term will be Web2 customers, because the actual Web3 customers who need AI are only a small part of this $2.7 trillion market (considering that BTC has half of the market share, and BTC itself does not need/use AI).

Web3’s AI use cases are just getting started, and it’s not clear how big its market will be. But one thing is intuitively certain - it will only be a part of the Web2 market for the foreseeable future. We believe that Web3 AI still has a bright future, but this means that the most common application of Web3 AI is currently serving Web2 customers.

Examples of Web2 customers that can benefit from Web3 AI include:

• Vertical industry software companies built from the ground up with AI at their core (such as Cedar.ai or Observe.ai)

• Large enterprises that fine-tune models for their own purposes (such as Netflix)

• Fast-growing AI providers (such as Anthropic)

• Software companies that add AI capabilities to existing products (such as Canva)

This is a relatively stable customer group because these customers are generally large and high-value. They are unlikely to go out of business anytime soon, and represent a very large potential customer base for AI services. Web3 AI services that serve Web2 customers will benefit from this stable customer base.

But why would a Web2 customer want to use the Web3 stack? The rest of this article will explain this rationale.

Practical Benefit 2: Lower GPU Usage Costs through GPU Decentralized Physical Infrastructure Networks (GPU DePINs)

GPU DePINs pool underutilized GPU computing power (the most reliable of which comes from data centers) and make these resources available for AI inference. Think of it simply as “Airbnb for GPUs” (i.e. collaborative consumption of underutilized assets).

The reason we are excited about GPU DePINs is as mentioned above, primarily because there are many GPU cycles being wasted right now due to the NVIDIA chip shortage, which could be used for AI inference. These hardware owners have incurred sunk costs and are currently underutilizing their equipment, so these partial GPU cycles can be provided at a lower cost than the status quo because it is effectively a "windfall" for the hardware owner.

Specific examples include:

1) AWS machines: If you rent an H100 from AWS today, you need to commit to renting it for at least one year because the market supply is tight. This leads to waste because you are unlikely to use your GPU 7 days a week, 365 days a year.

2) Filecoin mining hardware: The Filecoin network has a large subsidized supply, but the actual demand is not large. Unfortunately, Filecoin never found a real product-market fit, so Filecoin miners are in danger of going bankrupt. These machines are equipped with GPUs and can be repurposed for low-end AI inference tasks.

3) ETH Mining Hardware: When ETH moved from Proof of Work (PoW) to Proof of Stake (PoS), a large amount of hardware immediately became available, which can be repurposed for AI inference.

The GPU DePIN market is highly competitive, with multiple players offering products. For example, Aethir, Exabits, and Akash. Hack VC chose to support io.net, which also pools supply through partnerships with other GPU DePINs, so they currently support the largest supply of GPUs on the market.

It is important to note that not all GPU hardware is suitable for AI inference. One obvious reason is that older GPUs do not have enough GPU memory to handle large language models (LLMs), although there have been some interesting innovations in this regard. For example, Exabits has developed technology to load active neurons into GPU memory and inactive neurons into CPU memory. They predict which neurons need to be active/inactive. This allows AI workloads to be processed using low-end GPUs even when GPU memory is limited. This actually increases the usefulness of low-end GPUs for AI inference.

In addition, Web3 AI DePINs need to harden their products over time, providing enterprise-grade services such as single sign-on (SSO), SOC 2 compliance, service level agreements (SLAs), etc. This will be comparable to the cloud services currently enjoyed by Web2 customers.

Real Advantage #3: Non-censorship Model to Avoid OpenAI Self-censorship

There has been a lot of discussion about the issue of AI censorship. For example, Turkey once temporarily banned OpenAI (they later lifted the ban after OpenAI improved its compliance). We believe that this kind of national-level censorship is not fundamentally a concern, as countries need to embrace AI to remain competitive.

What's more interesting is that OpenAI will self-censor. For example, OpenAI will not process NSFW (not suitable for viewing in the workplace) content, nor will it predict the results of the next presidential election. We think there is an interesting and large market for AI applications that OpenAI is reluctant to touch for political reasons.

Open source is a good way to solve this problem, as a Github repository is not beholden to shareholders or a board of directors. An example is Venice.ai, which promises to protect user privacy and operate in a non-censorship manner. Of course, the key is its open source nature, which makes this possible. Web3 AI can effectively improve this by running these open source software (OSS) models on low-cost GPU clusters for inference. Because of this, we believe OSS + Web3 is an ideal combination to pave the way for non-censorship AI.

Real Benefit #4: Avoid Sending Personally Identifiable Information to OpenAI

Many large enterprises have privacy concerns about their internal corporate data. For these customers, it is difficult to trust a centralized third party like OpenAI to handle this data.

For these enterprises, using web3 may seem more scary because their internal data is suddenly on a decentralized network. However, there are already some innovations in privacy-enhancing technologies for AI:

Trusted Execution Environments (TEEs) like Super Protocol

Fully Homomorphic Encryption (FHE) like Fhenix.io (a portfolio company of funds managed by Hack VC) or Inco Network (both backed by Zama.ai) and Bagel’s PPML

These technologies are still evolving, and performance is improving with upcoming zero-knowledge (ZK) and FHE ASICs. But the long-term goal is to protect enterprise data while fine-tuning models. As these protocols emerge, web3 may become a more attractive place for privacy-preserving AI computation.

Real Benefit #5: Leverage the Latest Innovations in the Open Source Model

Over the past few decades, open source software (OSS) has been eroding the market share of proprietary software. We view LLM as a high-level proprietary software that is gradually becoming a disruptor to open source software. Some notable challengers include Llama, RWKV, and Mistral.ai. This list will undoubtedly grow over time (a more comprehensive list is available at Openrouter.ai). By leveraging web3 AI powered by the open source model, people can take full advantage of these new innovations.

We believe that an open source global development effort, combined with crypto incentives, can drive rapid innovation in the open source model and the agents and frameworks built on top of it over time. An example of an AI agent protocol is Theoriq. Theoriq leverages the open source model to create a network of composable and interconnected AI agents that can be assembled together to create more advanced AI solutions.

The reason we are so confident of this is because of past experience: most “developer software” has been surpassed by open source software over time. There is a reason why Microsoft, a proprietary software company, is now the most contributed Github company. If you look at how Databricks, PostGresSQL, MongoDB, etc. have disrupted proprietary databases, the entire industry is an example of being disrupted by open source software, so the precedent is quite strong here.

However, there is a small catch. One tricky thing about OSS LLMs is that OpenAI has begun signing paid data license agreements with organizations, such as Reddit and the New York Times. If this trend continues, it may become increasingly difficult for OSS LLMs to compete due to the economic barriers to accessing data. Nvidia may use confidential computing as a reinforcement tool for secure data sharing. Time will tell how this develops.

Real Benefit #6: Consensus via Costly Random Sampling or Zero-Knowledge Proofs

Verification is a challenge in web3 AI inference. It is possible for a validator to cheat on the results to earn fees, so verifying the inference is an important measure. It is important to note that although AI inference is still in its infancy, such cheating is inevitable unless measures are taken to weaken the incentive for such behavior.

The standard web3 approach is to have multiple validators repeat the same operation and compare the results. However, as mentioned earlier, AI inference is very expensive due to the current shortage of high-end Nvidia chips. Given that web3 can provide lower-cost inference through underutilized GPU DePINs, redundant computation will severely weaken the value proposition of web3.

A more promising solution is to perform zero-knowledge proofs for off-chain AI inference computations. In this case, a concise zero-knowledge proof can be verified to determine whether the model was trained correctly, or whether the inference was run correctly (called zkML). Examples include Modulus Labs and ZKonduit. The performance of these solutions is still in its infancy, as zero-knowledge operations require considerable computational resources. However, this may improve as zero-knowledge hardware ASICs become available in the near future.

A more promising idea is an approach to AI inference based on "optimistic" sampling. In this model, you only need to verify a small fraction of the results generated by the verifier, but set the economic cost high enough to punish verifiers who are caught cheating, so as to have a strong economic prohibitive effect. This way, you can save on redundant computation (see, for example, Hyperbolic's "Proof of Sampling" paper).

Another promising idea is a solution that uses watermarking and fingerprinting techniques, such as the one proposed by Bagel Network. This is similar to the mechanism that Amazon Alexa provides for quality assurance of its AI models on millions of devices.

Real Benefit #7: Savings through Composable OSS Stacks (OpenAI’s Profits)

The next opportunity web3 brings to AI is the democratization of cost reduction. So far, we’ve discussed ways to save on GPU costs through DePINs like io.net. However, web3 also offers opportunities to save on the profit margins of centralized web2 AI services (such as OpenAI, which has over $1 billion in annual revenue as of this writing). These cost savings come from using open source software (OSS) models instead of proprietary models, which results in additional cost savings because the model creators are not trying to make a profit.

Many OSS models will always be completely free, which provides the best economics for customers. However, there may also be some OSS models that try these monetization methods. Consider that only 4% of the models on Hugging Face are trained by companies with budgets to help subsidize these models (see here). The remaining 96% of the models are trained by the community. This 96% group of Hugging Face models face real costs (both computational and data). So these models need to be monetized somehow.

There are many proposals for monetizing this OSS model. One of the most interesting is the concept of an “Initial Model Offering” (IMO), where the model itself is tokenized, leaving a portion of the tokens to the team, and some of the model’s future revenue flows to token holders, although there are obviously some legal and regulatory hurdles to this.

Other OSS models will attempt to monetize based on usage. It’s important to note that if this were to become a reality, OSS models might start to look more and more like their web2 profit-generating counterparts. However, realistically, the market will be bifurcated, with some of these models being completely free.

Once you’ve chosen an OSS model, you can layer composable operations on top of it. For example, you can use Ritual.net for AI reasoning, and Theoriq.ai as an early leader in composable and autonomous on-chain AI agents (both backed by Hack VC).

Real Benefit #8: Decentralized Data Acquisition

One ​​of the biggest challenges facing AI is getting the right data for training models. We mentioned earlier that there are some challenges with decentralized AI training. But what about leveraging a decentralized network to acquire data that can then be used for training elsewhere, even on traditional web2 platforms?

That’s exactly what startups like Grass are doing (backed by Hack VC). Grass is a decentralized “data scraping” network where individuals contribute their machines’ idle processing power to fetch data for AI model training. In theory, at scale, this kind of data scraping could be superior to any one company’s in-house efforts because of the massive computing power of a large network of incentivized nodes. This includes not only getting more data, but also fetching it more frequently so that it’s more relevant and up-to-date. Since these data scraping nodes are inherently decentralized and not owned by a single IP address, it’s nearly impossible to stop this decentralized army of data scrapers. Additionally, they have a network of humans that clean and normalize the data to make it useful once it’s scraped.

Once you’ve acquired the data, you’ll also need an on-chain storage location, as well as LLMs (large language models) generated using that data. 0g.AI is an early leader in this regard. It’s a high-performance web3 storage solution optimized for AI that’s much cheaper than AWS (another economic success for Web3 AI), while also serving as data availability infrastructure for second layers, AI, and more.

It’s important to note that the role of data in web3 AI may change in the future. Currently, for LLMs, the status quo is to pre-train models using data and improve over time using more data. However, because data on the internet changes in real time, these models are always slightly out of date, so LLM reasoning is slightly inaccurate in response.

A new paradigm that may develop in the future is “live” data. The concept is that when the LLM is asked to make inferences, the LLM can use data by injecting it with data collected from the internet in real time. This way, the LLM will use the latest data. Grass is also working on this.

III. Conclusion

We hope this analysis is helpful to you as you think about the promise and reality of web3 AI. This is just a starting point for discussion, and the field is changing rapidly, so please feel free to join in and express your views, as we are willing to continue learning and building together.