So far, we have delved deeply into the behind-the-scenes of components and SDKs, as well as showcases that combine various components. This time, we will actually go hands-on to create our own component. In this article, we will try to create a component that actually collects data from the Web using Snapshot Indexer HTTPS.
In this project, let's create a Price Snapshot Indexer HTTPS that uses the Crypto API available on the Web to obtain the price of a specified currency. Beyond that, the data collected by this Snapshot Indexer can be used to create Oracle.
Completed Component Definition
Completed Component DefinitionThis time, let's create a Snapshot Indexer HTTPS that collects bitcoin prices. There are several APIs that provide bitcoin prices, but we will use coingecho's API to collect the current price. Let's construct Snapshot Indexer with a collection interval of one hour.
Prerequisite
The Chainsight CLI requires several tools and this CLI installation. Please refer to the following article for this preliminary preparation.
https://dev.to/hide_yoshi/step-by-step-creating-an-evm-price-oracle-with-chainsight-469g
Create a project
The first step is to build a project template. In this case, we only need one manifest, so let's proceed by adding a manifest of the type we want to create from an empty project. The command to create a project is csx new. Give the -no-sample option to make the project empty.
Looking at the created project, the manifest folder, components, is empty.
Now let's run csx add to add the manifest as instructed in the log.
Add a manifest
You can use csx add to interactively add a manifest of the type you wish to add. Let's actually run csx add with the name btc_snapshot_indexer and the type snapshot_indexer_https.
This will generate a manifest (.yaml) for btc_snapshot_indexer and update project.yaml to include btc_snapshot_indexer in the project itself.
Then update this generated manifest so that the bitcoin price can be retrieved. Get the bitcoin price by calling the /simple/price endpoint of the coingecko API.
https://www.coingecko.com/api/documentation
The parameters for actually taking this data are as follows. The id to retrieve bitcoin data is bitcoin. Let's also specify some parameters to extend the data available at this endpoint.
https://api.coingecko.com/api/v3/simple/price
?ids=bitcoin
&vs_currencies=usd,eur,btc,eth
&include_market_cap=true
&include_24hr_vol=true
&include_24hr_change=true
&include_last_updated_at=true
&precision=18
Modify datasource field to specify this parameter in the snapshot_indexer_https manifest. Specify the endpoint in the url field. Since the query parameters are static and immutable this time, set queries type to static and value field to the respective key and value.
datasource:
url: https://api.coingecko.com/api/v3/simple/price
headers:
Content-Type: application/json
queries:
type: static
value:
ids: bitcoin
vs_currencies: usd,eur,btc,eth
include_market_cap: true
include_24hr_vol: true
include_24hr_change: true
include_last_updated_at: true
precision: 18
The difference between the template generated by csx add and the updated manifest is as follows
9,10c9,11
< - DAI
< - USD
---
> - BTC
> - bitcoin
> - "Hands-on Sample"
18,19c19,25
< ids: dai
< vs_currencies: usd
---
> ids: bitcoin
> vs_currencies: usd,eur,btc,eth
> include_market_cap: true
> include_24hr_vol: true
> include_24hr_change: true
> include_last_updated_at: true
> precision: 18
Use this manifest to move on to the next step.
Generate a code
To generate the canister code for a component from its manifest, run csx generate with the project.yaml declaring the path to its manifest.
csx generate generates a project to generate the canister module under the src folder. If no customization is required, module generation can be performed immediately by running csx build in this state.
Since this Snapshot Indexer is intended to collect only bitcoin prices, not all data that can be retrieved from the API is required. Also, the more data content, the more expensive it is to retain, and the higher the probability of consensus generation failure when retrieving data via HTTPS outcalls. For more information, please refer to the following article.
Therefore, in order to avoid storing unnecessary data, modifications are made to narrow down the data to be collected.
To make customizations to the generated code, modify logics/(canister-name)/src/lib.rs in the src directory. For any type of canister, you can add your own customizations by modifying the file replacing canister-name with the name of the component of interest. If you want to know more about the project generated by csx generate, please refer to this article.
https://medium.com/@Chainsight_Network/behind-chainsight-how-modules-are-created-pt-1-32b13aa7f696
Now let's get back to the main issue. Let's open logics/btc_snapshot_indexer/src/lib.rs in your project. The code should look something like this
// Auto-generated code from manifest.
// You update the structure as needed.
// The existence of the SnapshotValue structure must be maintained.
use candid::{Decode, Encode};
#[derive(Debug, Clone, candid::CandidType, candid::Deserialize, serde::Serialize, chainsight_cdk_macros::StableMemoryStorable)]
pub struct SnapshotValue {
pub bitcoin: Bitcoin,
}
#[derive(Debug, Clone, candid::CandidType, candid::Deserialize, serde::Serialize, chainsight_cdk_macros::StableMemoryStorable)]
pub struct Bitcoin {
pub usd: f64,
pub usd_market_cap: f64,
pub usd_24h_vol: f64,
pub usd_24h_change: f64,
pub eur: f64,
pub eur_market_cap: f64,
pub eur_24h_vol: f64,
pub eur_24h_change: f64,
pub btc: f64,
pub btc_market_cap: f64,
pub btc_24h_vol: f64,
pub btc_24h_change: f64,
pub eth: f64,
pub eth_market_cap: f64,
pub eth_24h_vol: f64,
pub eth_24h_change: f64,
pub last_updated_at: i64,
}
This is an implementation for mapping the data obtained by calling the endpoints you specify. The trait you are using has the necessary functionality for the structure, so understand that that is the way it is and don't worry about it. The point of concern is the fields of the structure. Fields for a number of elements have already been declared. Where this code, which is implemented from the beginning, comes from is that the CLI automatically generates an implementation that matches the response from the endpoint you have set up. The actual response data will give you a sense of the implementation for mapping.
{
"bitcoin": {
"usd": 42451.59160065723,
"usd_market_cap": 832331572531.3098,
"usd_24h_vol": 12740914009.764158,
"usd_24h_change": 0.7450874696750942,
"btc": 1.0,
"btc_market_cap": 19611887.0,
"btc_24h_vol": 300160.8233960758,
"btc_24h_change": 0.0,
"eth": 18.650627839906598,
"eth_market_cap": 365799851.323311,
"eth_24h_vol": 5597576.829903464,
"eth_24h_change": 0.9213166048247731,
"last_updated_at": 1706500074
}
}
In this case, we only need to collect bitcoin price information. In other words, only bitcoin.usd data needs to be collected. Let's remove all but the usd field of the Bitcoin structure.
pub struct Bitcoin {
pub usd: f64,
- pub usd_market_cap: f64,
- pub usd_24h_vol: f64,
- pub usd_24h_change: f64,
- pub eur: f64,
- pub eur_market_cap: f64,
- pub eur_24h_vol: f64,
- pub eur_24h_change: f64,
- pub btc: f64,
- pub btc_market_cap: f64,
- pub btc_24h_vol: f64,
- pub btc_24h_change: f64,
- pub eth: f64,
- pub eth_market_cap: f64,
- pub eth_24h_vol: f64,
- pub eth_24h_change: f64,
- pub last_updated_at: i64,
}
Let's update the logic in this way and run csx generate again. You have the template logic generated once you run generate, and you have placed code that modifies it. Therefore, if you run csx generate again, you will see the log output that the execution has completed, skipping the logic generation.
Build a module
Generate a module from the code created in the previous processes. Let's run csx build immediately.
That's all it took to generate a module for Chainsight from your code! Let's actually check out the artifact.
The artifacts generated by csx build are placed in the artifacts folder. If you look in that folder, you will see WASM modules and candid interface files with the actual component names as their file names. For a more in-depth understanding, please refer to the following articles
https://medium.com/@Chainsight_Network/behind-chainsight-how-modules-are-created-pt-2-4cf29234b98b
That's all there is to the build. Now all you have to do is deploy and get the component up and running.
Deploy a component
Let's actually deploy the generated module in Chainsight. Components are deployed with csx deploy, and after deployment is complete, csx exec is executed to send a command to the component to start execution. Let's run through some options. Let's deploy to Internet Computer using --component to explicitly specify the component to process and --network to specify the network to run on.
csx deploy --network ic --component btc_snapshot_indexer
csx exec --network ic --component btc_snapshot_indexer
The deployment and operation start instruction is now complete! Now all you have to do is wait for the interval of the automatic execution and you can refer to the data.
Please take a look at the component that can be created in this hands-on session, which is actually running as a sample. (The description and execution interval have been slightly changed.) Using the Chainsight UI, you can check the acquired data, the execution cycle, and the state of the cycles that fuel it!
Source: https://beta.chainsight.network/explore/components/wqsbs-fyaaa-aaaal-qdezq-cai
This concludes our hands-on with Snapshot Indexer HTTPS. We will continue to create and publish new hands-on sessions in the future, so please look forward to them!