AVR8js High Memory and Slow... (benchmark ?!?!)

[ricardojlrufino]

Hi, I recently found an interesting project to emulate the AVR (Arduino) chip made entirely in javascript.
I thought it would be a way to be able to use and integrate with a java application.
However in the first tests with graaljs, I saw that I had a very high consumption of CPU and Memory, and slow:

Using: Ubuntu 18.04, Intel i7-4500, 16GB ram

/opt/graalvm-ce-java8-20.1.0/bin/node  --jvm FILE 
1gb of RAM, CPU starts high: 80%, CPU stabilize in: ~30%  (but slow execution) 

/opt/graalvm-ce-java8-20.1.0/bin/node  FILE 
500mb of RAM, CPU starts high: 70%, CPU stabilize in: ~25%  (but slow execution) 

Native:

node --version : v13.12.0
15mb of RAM, CPU: 25% (fast... )

Test:
https://github.com/ricardojlrufino/avr8js_demo_node

I found the difference in performance strange...
I think it can be an interesting benchmark for the graalVM

[ricardojlrufino]

I converted the scripts to pure javascript

-- using cmd line
/opt/graalvm-ce-java8-20.1.0/bin/js bundle.js

-- using java

package javascript.avr;

import java.io.IOException;
import java.net.URISyntaxException;
import java.nio.file.Files;
import java.nio.file.Paths;

import org.graalvm.polyglot.Context;
import org.graalvm.polyglot.Engine;
import org.graalvm.polyglot.HostAccess;

public class HelloPolyglot {
  
  /**
   * A GraalVM engine shared between multiple JavaScript contexts.
   */
  private  static final Engine sharedEngine = Engine.newBuilder().build();
  
  public static void main(String[] args) throws IOException, URISyntaxException {
    System.out.println("Hello Java!");
//     try (Context context = Context.create()) {
    try (Context context = Context.newBuilder()
        .allowExperimentalOptions(true)
        .allowHostAccess(HostAccess.ALL)
        .allowAllAccess(true)
        .allowHostClassLoading(true)
        .allowNativeAccess(true)
        .allowIO(true)
        .engine(sharedEngine)
//        .option("js.nashorn-compat", "true")
        .option("js.v8-compat", "true")
        .build()) {
      
      
      byte[] allBytes = Files.readAllBytes(Paths.get("/media/ricardo/Dados/TEMP/testes-java/avr8js_demo_vanilla/bundle.js"));
      context.eval("js", new String(allBytes)).putMember("fromJava", "true");
    }
  }
}

it's running on the browser a little faster

[wirthi]

HI @ricardojlrufino

thanks for your report. Can you please clarify what exactly you run, and what the output means:

I can run js bundle.js: this outputs something like CPU 11.593757125 - is that the performance (higher is better)? I see similar output on v8 but the number print does not seem to make sense, it continue to grows eternally it seems, is the value legitimate? I think I can see a deoptimization loop on our side here, will have to investigate.

Regarding your initial example node --jvm FILE - do you mean to do something like node --jvm run.js blink.hex? (run.js being the "main" file according to package.json). Gives me errors, both with and without --jvm (Failed: ReferenceError: print is not defined or Failed: TypeError: Access to host class java.lang.String is not allowed or does not exist.).

Ideal would be a commandline where I can 1:1 compare Graal.js and original Node.js. (so any commandline that works on original Node should work for us as well).

Thanks,
Christian

[wirthi]

Another finding: js bundle.js ultimately terminates after a few minutes with RangeError: Maximum call stack size exceeded - both on V8 and GraalVM.

[ricardojlrufino]

i'm improve the report, please see:
wokwi/avr8js#49 (comment)

check this branche:
https://github.com/ricardojlrufino/avr8js_demo_node/tree/vanilla

I can run js bundle.js: this outputs something like CPU 11.593757125 - is that the performance

now have a more clear and comparable output...

Regarding your initial example node --jvm FILE - do you mean to do something like ....

/opt/graalvm-ce-java8-20.1.0/bin/node run.js

[ricardojlrufino]

Can you please clarify what exactly you run

this code uses AVR8js, an Arduino processor simulator

It is running the following code (blink.hex)

    void setup() {
        Serial.begin(115200);
        pinMode(LED_BUILTIN, OUTPUT); // PIN 12
      }
      void loop() {
        digitalWrite(LED_BUILTIN, HIGH);
        Serial.println("V1");
        delay(100);
        digitalWrite(LED_BUILTIN, LOW);
        Serial.println("V0");
        delay(100);
      }

[ricardojlrufino]

Any ideia ? because the performance in the nodejs (from graalVM ) version will be different from the standard version ?

[wirthi]

Hi,

i am trying on your vanilla branch. Commit 3d27169389e015e5fa97892abf4a11b33ee97f8b

now have a more clear and comparable output...

Still get the same output of CPU [someNumber] when running graalvm/bin/js bundle.js

/opt/graalvm-ce-java8-20.1.0/bin/node run.js

Fails with TypeError: Access to host class java.lang.String is not allowed or does not exist.

/opt/graalvm-ce-java8-20.1.0/bin/node --jvm run.js

Fails with ReferenceError: print is not defined

Something is wrong with those commandlines, I don't think that's exactly how you ran them? After looking into your run.js: you have different codepaths for different engines, which is not great for a benchmark to have. And why DO you have a special path for Graal.js here at all? Graal.js is Node.js compatible, there is no necessity to detect the Java object and have a special case for this. Node.js mode will work fine on graalvm/bin/node.

Fixing that and trying again, I get:

with original Node.js 12.16.3:
Time: Blink finished in 105 ms

with GraalVM EE 20.1.0 in native mode graalvm/bin/node run.js
Time: Blink finished in 354 ms

with GraalVM EE 20.1.0 in jvm mode graalvm/bin/node --jvm run.js
Time: Blink finished in 198 ms

performance in the nodejs (from graalVM ) version will be different from the standard version?

Of course it will! In this mode, you are executing Node.js with a different JavaScript engine (GraalVM JavaScript instead of V8). This will of course show different performance characteristics. We are working on getting our peak performance as good as V8 - on some benchmarks, we can significantly outperform them, on many benchmarks we are still a bit slower. Like on this one, where Node.js/V8 is ~2x faster.

[ricardojlrufino]

you have different codepaths for different engines, which is not great for a benchmark to have

I was making the modifications and testing to understand how everything worked ... until then, I had not understood the differences from one to the other.

Your results seem to be better, if something related to 'EE' version ?!

We are working on getting our peak performance as good as V8

Great to know man.

I think I will follow an alternative to using a version of the AVR emulator in C