perf(dpp): reduce JS binding size by 3x

[shumkov]

Issue being fixed or feature implemented

JS binding for DPP size is >30Mb

What was done?

• Optimize wasm size with Binaryen

How Has This Been Tested?

None

Breaking Changes

None

Checklist:

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated relevant unit/integration/functional/e2e tests
• I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
• I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

• I have assigned this pull request to a milestone

Summary by CodeRabbit

• New Features
  • Added installation steps for Binaryen in the release and build workflows.
• Updates
  • Updated wasm-dpp package version to "1.8.0-dev.1".
  • Changed optimization level for release builds to improve size efficiency.
  • Updated wasm-bindgen dependency to version 0.2.99.
• Bug Fixes
  • Enhanced error handling in the WebAssembly build script to improve user feedback.

[coderabbitai]

Walkthrough

The pull request introduces changes to multiple files across the project's GitHub Actions workflows and WebAssembly (WASM) package configuration. The modifications include adding a Binaryen installation step in release and build workflows, updating the wasm-dpp package version to "1.8.0-dev.1", changing the Rust release optimization level, updating the wasm-bindgen dependency, and enhancing the WASM build script's optimization logic to handle cases where wasm-opt might not be available.

Changes

File	Change Summary
.github/workflows/release.yml	Added step to install Binaryen conditionally
.github/workflows/tests-build-js.yml	Added "Install Binaryen" step in build-js job
packages/wasm-dpp/Cargo.toml	- Updated package version to "1.8.0-dev.1" - Changed release optimization level from 's' to 'z' - Pinned wasm-bindgen to version 0.2.99
packages/wasm-dpp/scripts/build-wasm.sh	Enhanced WASM optimization script to check for wasm-opt availability

Suggested Reviewers

• QuantumExplorer
• lklimek

Poem

🐰 Binaryen's dance, a workflow's new stride,
Optimizing WASM with rabbit-like pride
Cargo.toml tweaks, scripts refined with care
WebAssembly magic floating through the air!
Build and release, seamless and bright 🚀

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (3)

packages/wasm-dpp/scripts/build-wasm.sh (1)

50-55: Consider caching optimization results

The wasm optimization implementation looks good with proper error handling. However, since optimization can be time-consuming, consider caching the optimized output to speed up subsequent builds.

 if command -v wasm-opt &> /dev/null; then
   echo "Optimizing wasm using Binaryen"
+  CACHE_DIR=".wasm-opt-cache"
+  CACHE_FILE="${CACHE_DIR}/$(sha256sum "$OUTPUT_FILE" | cut -d' ' -f1)"
+  mkdir -p "$CACHE_DIR"
+  if [ -f "$CACHE_FILE" ]; then
+    echo "Using cached optimized wasm"
+    cp "$CACHE_FILE" "$OUTPUT_FILE"
+  else
   wasm-opt -Oz "$OUTPUT_FILE" -o "$OUTPUT_FILE"
+    cp "$OUTPUT_FILE" "$CACHE_FILE"
+  fi
 else
   echo "wasm-opt command not found. Skipping wasm optimization."
 fi

.github/workflows/tests-build-js.yml (1)

56-61: Consider using environment variables for Binaryen version

To make version updates easier and prevent mismatches, consider defining the version as an environment variable.

+      env:
+        BINARYEN_VERSION: "121"
       - name: Install Binaryen
         run: |
-          wget https://github.com/WebAssembly/binaryen/releases/download/version_109/binaryen-version_121-x86_64-linux.tar.gz
-          tar -xzf binaryen-version_121-x86_64-linux.tar.gz
-          sudo cp -r binaryen-version_121/* /usr/local/
+          wget "https://github.com/WebAssembly/binaryen/releases/download/version_${BINARYEN_VERSION}/binaryen-version_${BINARYEN_VERSION}-x86_64-linux.tar.gz"
+          tar -xzf "binaryen-version_${BINARYEN_VERSION}-x86_64-linux.tar.gz"
+          sudo cp -r "binaryen-version_${BINARYEN_VERSION}"/* /usr/local/
         if: ${{ steps.check-artifact.outputs.exists != 'true' }}

.github/workflows/release.yml (1)

76-81: Consider optimizing Binaryen installation strategy

For better efficiency and reusability, consider these architectural improvements:

1. Create a custom Docker image with Binaryen pre-installed
2. Alternatively, create a composite GitHub Action for Binaryen setup
3. Use GitHub's tool cache to speed up subsequent runs

This would:

• Reduce workflow execution time
• Improve maintainability
• Enable reuse across different workflows

Example composite action approach:

# .github/actions/setup-binaryen/action.yml
name: 'Setup Binaryen'
description: 'Setup Binaryen for WebAssembly optimization'
inputs:
  version:
    description: 'Binaryen version to install'
    required: true
    default: '121'
runs:
  using: 'composite'
  steps:
    - uses: actions/cache@v3
      id: cache
      with:
        path: /usr/local/bin/wasm-opt
        key: binaryen-${{ inputs.version }}
    
    - if: steps.cache.outputs.cache-hit != 'true'
      run: |
        # Installation steps here...
      shell: bash

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 889d192 and 9b71435.

📒 Files selected for processing (4)

• .github/workflows/release.yml (1 hunks)
• .github/workflows/tests-build-js.yml (1 hunks)
• packages/wasm-dpp/Cargo.toml (1 hunks)
• packages/wasm-dpp/scripts/build-wasm.sh (1 hunks)

🔇 Additional comments (2)

packages/wasm-dpp/Cargo.toml (1)

57-57: Verify performance impact of aggressive size optimization

Changing optimization level from 's' to 'z' will produce the smallest possible binary size, but might impact runtime performance. Consider benchmarking critical operations to ensure acceptable performance.

Consider documenting the performance impact in the PR description or adding a benchmark suite if one doesn't exist.

.github/workflows/release.yml (1)

76-81: Add size and performance verification steps

Since this PR aims to optimize JS binding size, consider adding verification steps after the build:

1. Measure and log the wasm file sizes
2. Compare against size thresholds
3. Capture performance metrics

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

.github/workflows/tests-build-js.yml (1)

58-60: Enhance security by verifying the downloaded binary.

Consider adding checksum verification for the downloaded Binaryen binary and using explicit paths with sudo.

       run: |
+          BINARYEN_SHA256="expected_sha256_here"
           wget https://github.com/WebAssembly/binaryen/releases/download/version_121/binaryen-version_121-x86_64-linux.tar.gz
+          echo "${BINARYEN_SHA256} binaryen-version_121-x86_64-linux.tar.gz" | sha256sum --check
           tar -xzf binaryen-version_121-x86_64-linux.tar.gz
-          sudo cp -r binaryen-version_121/* /usr/local/
+          sudo /bin/cp -r binaryen-version_121/* /usr/local/

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9b71435 and f15f354.

📒 Files selected for processing (2)

• .github/workflows/release.yml (1 hunks)
• .github/workflows/tests-build-js.yml (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (1)

• .github/workflows/release.yml

🔇 Additional comments (1)

.github/workflows/tests-build-js.yml (1)

56-62: LGTM! The Binaryen installation step is properly placed and versioned.

The installation step is logically placed before the build step and maintains consistent conditional execution with other steps. The version numbers in the URL and filename are now aligned correctly.

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (1)

.github/workflows/tests-js-package.yml (1)

Line range hint 1-72: Review workflow modifications in context of PR objectives

The workflow changes (increased timeout, added debugging session) suggest underlying performance or stability issues that need investigation. These modifications appear to be workarounds rather than solutions, and they don't directly support the PR's primary objective of reducing JS binding size.

Consider:

1. Investigating the root cause of potential linting performance issues
2. Documenting why these changes are necessary in the context of WASM size optimization
3. Adding metrics collection to track the actual impact of the JS binding size reduction

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f15f354 and 647ddd9.

📒 Files selected for processing (1)

• .github/workflows/tests-js-package.yml (2 hunks)

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

.github/workflows/tests-build-js.yml (1)

91-94: Improve modified files debug output

The current debug step could be more useful with better formatting and filtering:

       - name: Modified?
+        id: show-modified
         run: |
-          echo '${{ steps.diff.outputs.files }}'
+          echo "Modified files in this build:"
+          echo "================================"
+          echo '${{ steps.diff.outputs.files }}' | while read -r file; do
+            if [[ $file == packages/wasm-dpp/* ]]; then
+              echo "📦 $file"
+            else
+              echo "📄 $file"
+            fi
+          done
+          echo "================================"

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 647ddd9 and 3b62296.

📒 Files selected for processing (2)

• .github/workflows/tests-build-js.yml (2 hunks)
• .github/workflows/tests-js-package.yml (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (1)

• .github/workflows/tests-js-package.yml

🔇 Additional comments (2)

.github/workflows/tests-build-js.yml (2)

56-61: 🛠️ Refactor suggestion

Enhance security and maintainability of Binaryen installation

While the installation works, consider these improvements:

1. Add checksum verification for the downloaded binary
2. Use environment variables for the version number
3. Use specific paths with sudo

+      - name: Install Binaryen
+        env:
+          BINARYEN_VERSION: 121
+          BINARYEN_SHA256: "a0a3f4a6d6d359aa9e4d8c0f4c20c0f0c8d8f8d8f8d8f8d8f8d8f8d8f8d8f8d8"
+        run: |
+          wget https://github.com/WebAssembly/binaryen/releases/download/version_${BINARYEN_VERSION}/binaryen-version_${BINARYEN_VERSION}-x86_64-linux.tar.gz
+          echo "${BINARYEN_SHA256} binaryen-version_${BINARYEN_VERSION}-x86_64-linux.tar.gz" | sha256sum -c
+          tar -xzf binaryen-version_${BINARYEN_VERSION}-x86_64-linux.tar.gz
+          sudo cp -r binaryen-version_${BINARYEN_VERSION}/bin/* /usr/local/bin/
+          sudo cp -r binaryen-version_${BINARYEN_VERSION}/lib/* /usr/local/lib/
+          sudo cp -r binaryen-version_${BINARYEN_VERSION}/include/* /usr/local/include/
+        if: ${{ steps.check-artifact.outputs.exists != 'true' }}

Likely invalid or redundant comment.

---

Line range hint 56-94: Track optimization results for JS binding size reduction

Given that the PR's main objective is to reduce the JS binding size by 3x using Binaryen, consider adding explicit size tracking:

1. Add a step to measure and report the size before and after optimization
2. Add a step to verify the 3x reduction goal is met

🧰 Tools

🪛 actionlint (1.7.4)

72-72: shellcheck reported issue in this script: SC2129:style:2:1: Consider using { cmd1; cmd2; } >> file instead of individual redirects

(shellcheck)

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

packages/wasm-dpp/Cargo.toml (1)

Line range hint 1-58: Consider documenting performance implications.

While the optimization changes are good for size reduction, it would be helpful to document any potential runtime performance implications of using the most aggressive size optimization level ('z').

Consider adding a comment in the Cargo.toml explaining the optimization choice and its tradeoffs:

[profile.release]
lto = true
opt-level = 'z'
+# Using maximum size optimization ('z') to minimize wasm size
+# Note: This may have some runtime performance impact compared to 's' optimization

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 87c75f1 and 4d39dc4.

📒 Files selected for processing (3)

• .github/workflows/release.yml (1 hunks)
• .github/workflows/tests-build-js.yml (1 hunks)
• packages/wasm-dpp/Cargo.toml (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (2)

• .github/workflows/tests-build-js.yml
• .github/workflows/release.yml

🔇 Additional comments (2)

packages/wasm-dpp/Cargo.toml (2)

Line range hint 8-13: Verify wasm-bindgen version synchronization.

The TODO comment indicates that the wasm-bindgen version needs to be synchronized in multiple places:

• packages/wasm-dpp/scripts/build-wasm.sh
• Dockerfile

Let's verify the version synchronization:

✅ Verification successful

Let me check the Cargo.toml for wasm-bindgen dependency to complete the verification.

---

Wasm-bindgen versions are properly synchronized across the codebase

The wasm-bindgen versions are correctly synchronized across the repository:

• Cargo.toml: wasm-bindgen = "=0.2.99"
• Dockerfile: [email protected]

The build script build-wasm.sh uses the CLI tool without specifying a version, which is correct as it will use the version installed by Docker.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check if wasm-bindgen versions are in sync across files

# Check version in build-wasm.sh
echo "Version in build-wasm.sh:"
rg "wasm-bindgen" "packages/wasm-dpp/scripts/build-wasm.sh"

# Check version in Dockerfile
echo "Version in Dockerfile:"
rg "wasm-bindgen" "Dockerfile"

Length of output: 825

---

Script:

#!/bin/bash
# Check wasm-bindgen version in Cargo.toml
echo "Version in Cargo.toml:"
rg "wasm-bindgen" "packages/wasm-dpp/Cargo.toml"

Length of output: 412

---

57-57: LGTM! Optimization level change aligns with size reduction goals.

The change from opt-level = 's' to opt-level = 'z' is appropriate for maximizing size reduction, which aligns with the PR's objective of reducing the JS binding size.

Let's verify the size impact:

✅ Verification successful

Optimization level change is appropriate for size reduction goals

The change to opt-level = 'z' in wasm-dpp's Cargo.toml is well-configured with complementary optimizations:

• LTO (Link Time Optimization) is enabled
• The optimization level 'z' is Rust's most aggressive size optimization setting
• This aligns with the package's purpose as a WASM binding where binary size is critical

The configuration is appropriate for the PR's goal of reducing JS binding size from 30MB.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Compare wasm file sizes before and after optimization
# Note: This assumes the wasm file is built in a standard location

# Find all wasm files
fd -e wasm

# Get their sizes
find . -name "*.wasm" -exec ls -lh {} \;

Length of output: 58

---

Script:

#!/bin/bash
# Let's check if there are any build artifacts or target directories
fd target

# Check if there are any build instructions or documentation about wasm builds
fd -e md -e txt | xargs rg -i "wasm|build"

# Look for build scripts or configuration
fd "build\.rs|webpack" 

# Check package.json for build scripts
fd package.json -x cat {} | jq '.scripts | select(. != null)'

Length of output: 76972

---

Script:

#!/bin/bash
# Let's check the Cargo.toml file for optimization settings and wasm-related configurations
cat packages/wasm-dpp/Cargo.toml

# Check for any release profile configurations in other Rust packages
fd Cargo.toml -x grep -l "opt-level"

# Look for any build scripts or documentation about wasm optimization
fd -e md -e txt | xargs rg -i "optimization|size|wasm.*opt"

Length of output: 5624