Feb 5, 2024

How Chainsight Component Modules are Created Part.2, Building Modules

by Megared

This article is part 2 of the "Behind Chainsight: How Modules are Created" series. In the previous article, we introduced the code generation process as the first step in the module generation process.

https://chainsight-lp-git-draft-horizonx-tech.vercel.app/blog/how-modules-are-created-pt1-generate-source

chainsight_deepdive_into_showcase-process_create-build-2.png

Users can generate contract code simply by writing a YAML manifest. So what do we do from here to make a viable WASM module? Let's take a quick look.

Behind Chainsight CLI: Build Command

The contract code was generated by csx generate. Use csx build to generate actual deployable WASM modules from these codes.

The csx build follows the Rust build process of dfx build from dfx, DFINITY's stock CLI, but adds its own shrink/optimize and metadate process to it. Shrinkage/optimization is performed using ic-wasm 's API and does not require any individual configuration by the user.

https://github.com/dfinity/ic-wasm

Regarding the process of granting metadata, static data for users from labels, tags, etc. set in the manifest will be incorporated into the WASM module.

metadata:
  label: Sample Snapshot Indexer Evm
  type: snapshot_indexer_evm
  tags:
  - ERC-20
  - Ethereum
  - DAI
...
% ic-wasm artifacts/sample_snapshot_indexer_evm.wasm metadata
icp:public chainsight:label
icp:public chainsight:component_type
icp:public chainsight:description
icp:public chainsight:tags
icp:public chainsight:sources
icp:public chainsight:intervalSec

% ic-wasm artifacts/sample_snapshot_indexer_evm.wasm metadata chainsight:label
Sample Snapshot Indexer Evm
% ic-wasm artifacts/sample_snapshot_indexer_evm.wasm metadata chainsight:component_type
snapshot_indexer_evm

Chainsight's own metadata is discussed in detail in the following article.

Technical Guide: Understanding Component Metadata | by Chainsight | Jan, 2024 | Medium

This is the end of the module construction. While users do not need to be directly aware of this, let's look a little further at the processing provided by the CLI beyond that.

With these processes, deployable WASM modules have been created for each component. However, there is still more required to deploy them. First is the Candid file that contains the interface information of those WASM modules. Next, you will need dfx.json, a configuration file for Internet Computer that incorporates the settings to use them. If users want to deploy Internet Computer contracts on their own, they must build them manually. However, csx build automatically generates these as well, and finally creates a workspace (folder) that can be deployed on its own, with no dependencies anywhere.

pj_root/artifacts
|- component_1.did // Candid interface for the component
|- component_1.wasm // Component WASM module
|- component_2.did
|- component_2.wasm
...
|- component_N.did
|- component_N.wasm
L dfx.json // Config of the project when deploying to either local or ICP

The generic dfx.json is linked to the Motoko and Rust code projects.

// General example (not Chainsight)
{
  "version": 1,
  "canisters": {
    "example_motoko": {
      "type": "motoko",
      "main": "src/canisters/motoko/Main.mo",
    },
    "example_rust": {
      "type": "rust",
      "candid": "src/canisters/rust/interface.did",
      "package": "example_rust"
    },
	},
  ...
}

This is not a bad configuration, but it is linked to the code project itself, creating a dependency with the code folders and packages (in effect, a dependency with the code itself), and the amount of resources that are dependent on it is enormous. Chainsight uses the WASM modules generated by the aforementioned build process to provide a dfx configuration file with no dependencies other than the cut-out WASM modules and Candid interface.

// by Chainsight CLI
{
  "version": 1,
  "canisters": {
    "component_1": {
      "type": "custom",
      "candid": "./component_1.did",
      "wasm": "./component_1.wasm",
      "metadata": [
        {
          "name": "candid:service",
          "visibility": "public"
        }
      ]
    },
    ...
  },
  ...
}

In this way, the Chainsight project folder has as few dependencies as possible, and an area is reserved for each purpose to extract the work or parts of it.

chainsight_deepdive_into_showcase-process_folder-by-build.png

We can take advantage of some of this activity by separating not only the composition of the project itself, but also the generation process itself for generating and deploying modules. For example, the build artifact itself can be extracted and used elsewhere. Since the WASM module itself can be retrieved, it can be tested against WASM itself, commanded to update as a module, and so on.

Foundation for behaving as a CLI

So far, the explanation has been based on the processing process provided by the CLI. As an extra topic, we will briefly introduce the core technology upon which the CLI itself is built. The basic premise is that the Chainsight CLI is built entirely in the Rust programming language.

https://github.com/horizonx-tech/chainsight-cli

Rust has an excellent library for creating CLI tools.

The clap library is used to provide a variety of commands.

https://github.com/clap-rs/clap

csx --help
Chainsight command-line execution envirionment

Usage: csx [OPTIONS] <COMMAND>

Commands:
  new       Generates Chainsight project with built-in templates
  add       Generates component manifest of specified type and adds to your project
  generate  Generate codes according to project/component manifests [aliases: gen]
  build     Builds your project to generate canisters' modules for Chainsight
  deploy    Deploy the components of your project. If you want to operate on a local network, you need to build a local dfx network in advance
  exec      Calls for component processing. Currently supports initialization and task start instructions
  remove    Remove your Chainsight project. Before this operation, you must delete your canisters in local/ic nodes
  delete    Delete your Chainsight component. This command deletes the component with sidecars and allows you to recover the remaining cycles
  help      Print this message or the help of the given subcommand(s)

Options:
  -v, --verbose...  Displays detailed information about operations. -vv will generate a very large number of messages and can affect performance
  -q, --quiet...    Suppresses informational messages. -qq limits to errors only; -qqqq disables them all
  -h, --help        Print help
  -V, --version     Print version

Provides excellent log output using the slog library.

https://github.com/slog-rs/slog

% csx generate
Jan 11 02:07:33.797 INFO Start code generation for project 'sample_pj'
Jan 11 02:07:33.797 INFO Load env file: "./.env"
Jan 11 02:07:33.801 INFO [sample_indexer] Start processing...
Jan 11 02:07:33.801 INFO [sample_indexer] Skip creating logic project: './src/logics/sample_indexer' already exists
Jan 11 02:07:33.802 INFO [sample_indexer] Generate interfaces (.did files) ...
Jan 11 02:07:35.248 INFO [sample_indexer] Succeeded: Generate interfaces (.did files)
Jan 11 02:07:35.248 INFO [sample_lens] Start processing...
Jan 11 02:07:35.249 INFO [sample_lens] Skip creating logic project: './src/logics/sample_lens' already exists
Jan 11 02:07:35.249 INFO [sample_lens] Generate interfaces (.did files) ...
Jan 11 02:07:36.544 INFO [sample_lens] Succeeded: Generate interfaces (.did files)
Jan 11 02:07:36.544 INFO [sample_relayer] Start processing...
Jan 11 02:07:36.544 INFO [sample_relayer] Skip creating logic project: './src/logics/sample_relayer' already exists
Jan 11 02:07:36.545 INFO [sample_relayer] Generate interfaces (.did files) ...
Jan 11 02:07:37.720 INFO [sample_relayer] Succeeded: Generate interfaces (.did files)
Jan 11 02:07:37.720 INFO Project 'sample_pj' codes/resources generated successfully

It also provides other interactive instructions and generic error handling.

https://github.com/console-rs/dialoguer

https://github.com/dtolnay/anyhow

With the help of these ecosystems, Chainsight CLI is able to provide useful functionality to its users.


In this article, we have focused on the build processes behind the scenes of the Chainsight CLI.  Throughout the series, we have delved into the process of building a module from empty. Thank you for your patience throughout.

We still have the innovation and technology behind the scenes for Chainsight to work as a Platform. See you again in the next article.

Follow us on Twitter and Medium for updates! We look forward to seeing you there!

Written by Megared@Chainsight