Jan 18, 2024

Chainsight Showcase Description, Price Oracle with HTTPS outcalls, a stable integration with the Web in Internet Computer

by Megared

In our last issue, we covered Price Oracle in our first Deep Dive into Showcase post. This time, we will deal with the same subject matter, Price Oracle, but we would like to present a Showcase using a different means. Specifically, we will use a different type of Snapshot Indexer. Previously, we used Snapshot Indexer EVM, a component that allows integration into EVM-compatible chains, and this time we will use Snapshot Indexer HTTPS, a component that allows integration into EVM-compatible blockchains.

This Price Oracle is built from a single Snapshot Indexer HTTPS, Algorithm Lens, and Relayer, for a total of three Components.

chainsight_deepdive_into_showcase-price_oracle_with_https.png

The role of each Component is the same as in the previous Price Oracle, with the only difference being the type of Snapshot Indexer used.

https://github.com/horizonx-tech/chainsight-showcase/tree/main/price_feed/price_oas

In the previous article, I introduced Snapshot Indexer at the code level, but I did not mention what types of Snapshot Indexer are available. Currently, the following types are implemented.

Snapshot Indexer EVM: Fetches data to EVM-compatible blockchain

Snapshot Indexer ICP: Fetches data to a canister on an Internet Computer

Snapshot Indexer HTTPS: Fetch data to any resource/host on the Web

This time we will use CoinGecko's API and CoinMarketCap's API with Snapshot Indexer HTTPS.

Dig deeper into Snapshot Indexer HTTPS

In the first place, it is generally not possible to access the general Web world (Web2) from the Blockchain world (Web3). The blockchain world is decentralized and processing there is deterministic, whereas the general web world is unstable and processing results can be non-deterministic, usually accessed centrally controlled (the points listed here are representative and partial). But on the other hand, there is a lot of useful data out there in the general web world. Oracle, which now bridges to those data with the blockchain, has yet to cover it all.

Internet Computer has made this common Web communication possible with a single Contract (Canister on IC). HTTPS outcalls make this possible, and while there are several technical features of HTTPS outcalls for interfacing with the Web, we will focus on just one here. I told you earlier that the blockchain world is deterministic, while the web world is unstable and potentially non-deterministic. HTTPS outcalls resolve this. Distributed nodes (i.e., multiple replicas) that build the IC make the same request and reach consensus on the processing result. In other words, it guarantees that the request will yield the same request result.

Untitled.png

Source: https://support.dfinity.org/hc/en-us/articles/12342341722260-What-is-an-HTTPS-Outcall-

For additional information, please check the links below.

https://internetcomputer.org/https-outcalls

https://wiki.internetcomputer.org/wiki/HTTPS_outcalls

https://internetcomputer.org/docs/current/developer-docs/integrations/https-outcalls/

Snapshot Indexer HTTPS takes advantage of this HTTPS outcalls to provide a datastore that uses resources from the Web as its data source.

How to use Snapshot Indexer HTTPS

Snapshot Indexer HTTPS periodically retrieves data from the specified endpoints and stores it within Component. This PriceOracle component periodically calls functions to retrieve the latest prices from CoinGecko's API and CoinMarketCap's API.

Below is the Manifest for implementing the above Snapshot Indexer HTTPS for Price Oracle.

datasource:
  url: <https://api.coingecko.com/api/v3/simple/price>
  headers:
    Content-Type: application/json
  queries:
    type: static
    value:
      ids: oasys
      vs_currencies: usd
      include_24hr_vol: true
      precision: 18

The destination of the call is set in the datasource field. The url specifies the base endpoint, headers specifies the header, and queries specifies the parameters to be specified in the query string. In this case, these definitions result in the following URL for the request.

https://api.coingecko.com/api/v3/simple/price?ids=oasys&vs_currencies=usd&include_24hr_vol=true&precision=18

Snapshot Indexer HTTPS currently supports only GET requests.

The structure of the data obtained by this request is also automatically generated, and its definition can be customized by the user. The following data was obtained from the endpoint at which this request was made,

{
  "oasys": {
    "usd": 0.11314642006361941,
    "usd_24h_vol": 3073098.927703896
  }
}

The following structure is automatically generated from it. Chainisight is able to perform automatic code generation up to this point.

pub struct SnapshotValue {
    pub oasys: Inner,
}
pub struct Inner {
    pub usd: f64,
    pub usd_24h_vol: f64,
}

This is all it takes to bring data from the Web world into the Blockchain world with consensus building.

In order to actually perform HTTPS outcalls on an Internet computer, one must write code from scratch that implements the HTTPS outcalls themselves from the dfinity library, adds an implementation to process the response from the specified destination, adds filters to the response, manages Cycles, and so on. Cycles management and many other considerations must be written from scratch.

https://github.com/dfinity/examples/blob/master/rust/send_http_get/src/send_http_get_backend/src/lib.rs

In Chainsight, users do not need to write the above code, and it can be achieved with zero code (Manifest only).

What if the response to a given API is highly variable (e.g., dependent on execution time) or the data size is very large?

If the responses to the same request by multiple replicas made in a single HTTPS outcall are slightly different, the request will fail due to lack of consensus. Also, if the data size is huge, even if consensus is successfully reached, the gas consumption will be very high due to the large data size handled in the communication. (This is especially stressful when these are brought to you from data fields that you do not need.)

To avoid this, fields that are not needed can be eliminated from the actual response, thus reducing the final data size without affecting consensus.

Chainsight can perform the above actions simply by excluding the parameter in the response type declaration.

pub struct SnapshotValue {
    pub oasys: Inner,
}
pub struct Inner {
    pub usd: f64,
    // pub usd_24h_vol: f64,
}

Chainsight's Snapshot Indexer HTTPS takes full advantage of HTTPS outcalls in Internet Computer to provide a user-friendly format with minimal effort.

The Showcase Price Oracle includes examples using Snapshot Indexer EVM and Snapshot Indexer HTTPS, Users can use these as a reference to create an EVM-compatible, arbitrary Price Oracle with the Web as the data source.


In this article, I introduced the second Price Oracle example, which uses the Web as its data source. I also explained Internet Computer's HTTPS outcalls and Chainsight's Snapshot Indexer HTTPS, which are the foundation. So stay tuned for the sequel to Deep Dive into Showcase!

Follow us on Twitter and Medium for updates! We look forward to your participation.

Written by Megared@Chainsight