All You Need to know about Blockchain Indexers
Published
One of the most useful tool in Decentralized Finance and everything Blockchain is the Indexer. If you’ve ever checked on any sort of analytics or historical data for a smart contract, there’s a good chance an indexer was used at some point to collect the relevant information.
Let’s see what’s an indexer and why it is so necessary.
What’s an indexer?
While it is possible to just query data straight away from a blockchain, this process is quite inefficient and cumbersome and is best explained through a simple example:
Let’s say I want to obtain the 10 latest transactions of an address (we’ll call it Alice as is tradition). To do so, we quickly end up facing a wall: We just don’t know which blocks include transactions that were made by Alice.
This means that we’d need to go from the current block height, iterate down and query every block trying to find these transactions. This is clearly not scalable as you would have to repeat the same process for every single query and that’s where an indexer comes into play.
Simply put, an indexer is a tool that will crawl through a blockchain and scan every single block in order to extract the data it contains. This data can then be stored in a consistent and easy to query format, most likely in a database of some sort.
Getting away from the “per block” behaviour of the Blockchain allows you to search for specific attributes in a much more efficient way: Want to get transactions that are relevant to a specific smart contract and took place during the American day time? This suddenly becomes very doable.
An example: NFT Marketplace
NFT stands for Non Fungible Tokens, meaning tokens that have a specific value that makes them unique inside a collection of them. You might have heard about them as they’ve made quite a splash in recent years with some Ape drawings reaching an outrageous market value.
Anyways, an NFT marketplace faces a series of challenges in order to present all the relevant data it requires. Here’s a short list of what such a marketplace usually needs to know:
- Who owns an NFT
- How much did this NFT sell for in the past
- Keep track of every order relevant to a token or to a collection
- Keep track of historical trades
- Keep track of the “metadatas” of an NFT and its collection (image, sound, royalties and so on)
Imagine having to query the blockchain on the fly for all this info. There’s no way to provide an efficient experience for anyone visiting your marketplace if you need to spend minutes processing the data required for a user.
Next up, writing an indexer
I’m currently writing an indexer for a Cosmos blockchain in Rust, I’ll explain how it’s usually done and show off a basic API and parsing loop.