JNI Symbol Conflicts in Mac OS

Background

When I was adding Mac OS support to my open source project Javet, I found a weird behavior caused by JNI symbol conflicts which resulted in JVM 8 core-dump.

Javet is Java + V8 (JAVa + V + EighT). It is an awesome way of embedding Node.js and V8 in Java. It's embedding behavior relies on 2 JNI libraries one for Node.js and one for V8. Both of them are loaded by dedicated classloader and share the same API and symbols, except that the Node.js one exposes more Node.js flavored symbols.

In Javet unit test, it loads both Node.js and V8 libraries and performs complicated test. The test goes well in Linux and Windows, but core-dumps in Mac OS.

Analysis

The dump file shows the call stack crosses the library boundary.

...node...dylib
...node...dylib
...node...dylib <== Something wrong happens here.
...v8...dylib
...v8...dylib
...v8...dylib

It seems JVM in Mac OS registers global symbols in the same memory space, so that the V8 library calls into Node.js library. Obviously, that for sure leads to core-dump.

Failed Attempts

• I tried various distributions of JDK 8. None of them worked. It seems to be the Mac OS' behavior.
• I tried to adjust the CMakeLists.txt. All my attempts failed.

Fix

I kept reviewing the call stack and realized the issue might be gone if these symbols were not global symbols.

So I created jni/exported_symbols_list.txt with the following content.

_JNI_OnLoad
_JNI_OnUnload
_Java_com_caoccao_javet_*
_napi_*

Then, I updated CMakeLists.txt with the following content.

target_link_libraries(Javet PUBLIC -exported_symbols_list ${CMAKE_SOURCE_DIR}/jni/exported_symbols_list.txt)

And, it works, no more core-dump with all test cases pass.

Javet - Java and JavaScript Interop

Javet is Java + V8 (JAVa + V + EighT). It is an awesome way of embedding Node.js and V8 in Java.

From v0.9.8, Javet allows injecting arbitrary Java objects into V8 which enables the complete interop between Java and JavaScript. To enable this feature, application just needs to call v8Runtime.setConverter(new JavetProxyConverter());. Here are 3 examples.

Inject a Static Class

v8Runtime.getGlobalObject().set("System", System.class);
v8Runtime.getExecutor("function main() {\n" +
        // Java reference can be directly called in JavaScript.
        "  System.out.println('Hello from Java');\n" +
        // Java reference can be directly assigned to JavaScript variable.
        "  const println = System.out.println;\n" +
        // Java reference can be directly assigned to JavaScript variable.
        "  println('Hello from JavaScript');\n" +
        "}\n" +
        "main();").executeVoid();
v8Runtime.getGlobalObject().delete("System");

/*
 * Output:
 *   Hello from Java
 *   Hello from JavaScript
 */

Inject an Enum

v8Runtime.getGlobalObject().set("Color", Color.class);
System.out.println(v8Runtime.getExecutor("Color.pink.toString();").executeString());
System.out.println("The enum in JavaScript is the one in Java: " +
        (Color.pink == (Color) v8Runtime.getExecutor("Color.pink;").executeObject()));
v8Runtime.getGlobalObject().delete("Color");

/*
 * Output:
 *   java.awt.Color[r=255,g=175,b=175]
 *   The enum in JavaScript is the one in Java: true
 */

Inject a Pattern

Pattern pattern = Pattern.compile("^\\d+$");
v8Runtime.getExecutor("function main(pattern) {\n" +
        "  return [\n" +
        "    pattern.matcher('123').matches(),\n" +
        "    pattern.matcher('abc').matches(),\n" +
        "  ];\n" +
        "}").executeVoid();
System.out.println(v8Runtime.getGlobalObject().invokeObject("main", pattern).toString());

/*
 * Output:
 *   [true, false]
 */

Inject a StringBuilder

v8Runtime.getGlobalObject().set("StringBuilder", StringBuilder.class);
 System.out.println(v8Runtime.getExecutor("function main() {\n" +
         "  return new StringBuilder('Hello').append(' from StringBuilder').toString();\n" +
         "}\n" +
         "main();").executeString());
 v8Runtime.getGlobalObject().delete("StringBuilder");

/*
 * Output:
 *   Hello from StringBuilder
 */

Peking Opera Blues: Pure Delirium

Early today I was watching the newly (in 2020) released extras in a 2-disc BD and was shocked by your 18'25 story with Peking Opera Blues. At the end of it, I found a link to this blog post. What an amazing thing from watching BD extras.

Javet - Patch V8 Function at Source Code Level

Javet is Java + V8 (JAVa + V + EighT). It is an awesome way of embedding Node.js and V8 in Java.

With the release of v0.8.8, Javet supports patching V8 function at source code level on the fly.

Why is That Important?

Functions can be changed on the fly at JavaScript code level via Javet API. Why to choose this approach? Because sometimes local scoped context is required which is usually called closure. E.g:

const a = function () {
    const b = 1;
    return () => b;
}
const x = a();
console.log(x());
// Output is: 1

Local const b is visible to the anonymous function at line 3, but invisible to the function interceptor. Javet provides a way of changing the function at JavaScript source code level so that local scoped context is still visible.

How?

getSourceCode() and setSourceCode(String sourceCode) are designed for getting and setting the source code. setSourceCode(String sourceCode) actually performs the follow steps.

def setSourceCode(sourceCode):
    existingSourceCode = v8Function.getSourceCode()
    (startPosition, endPosition) = v8Function.getPosition()
    newSourceCode = existingSourceCode[:startPosition] + sourceCode + existingSourceCode[endPosition:]
    v8Function.setSourceCode(newSourceCode)
    v8Function.setPosition(startPosition, startPosition + len(sourceCode))

Be careful, setSourceCode(String sourceCode) has radical impacts that may break the execution because all functions during one execution share the same source code but have their own positions. The following diagram shows the rough memory layout. Assuming function (4) has been changed to something else with position changed, function (1) and (2) will not be impacted because their positions remain the same, but function (3) will be broken because its end position is not changed to the end position of function (4) accordingly.

Javet does not scan memory for all impacted function. So, it is caller's responsibility for restoring the original source code after invocation. The pseudo logic is as following.

originalSourceCode = v8ValueFunction.getSourceCode()
v8ValueFunction.setSourceCode(sourceCode)
v8ValueFunction.call(...)
v8ValueFunction.setSourceCode(originalSourceCode)

Why does setSourceCode() sometimes return false? Usually, that means the local scoped context hasn't been generated by V8. getJSScopeType().isClass() == true indicates that state. After callVoid(null), the local scoped context will be created with getJSScopeType().isFunction() == true and setSourceCode() will work. The pseudo logic is as following.

originalSourceCode = v8ValueFunction.getSourceCode()
if (v8ValueFunction.getJSScopeType().isClass()) {
    try {
        v8ValueFunction.callVoid(null);
        // Now v8ValueFunction.getJSScopeType().isFunction() is true
    } catch (JavetException e) {
    }
}
v8ValueFunction.setSourceCode(sourceCode) // true
v8ValueFunction.call(...)
v8ValueFunction.setSourceCode(originalSourceCode)

The rough lifecycle of a V8 function is as following.

What is the Source Code of a Function in V8?

When V8 calculates start position of a function, it does not include the keyword function and function name. E.g.

function abc(a, b, c) { ... } // Source code is (a, b, c) { ... }

(a, b, c) => { ... }          // Source code is (a, b, c) => { ... }

So, please always discard the keyword function and function name when calling setSourceCode().

Javet — An awesome way of embedding Node.js and V8 in Java

Javet is Java + V8 (JAVa + V + EighT). It is an awesome way of embedding Node.js and V8 in Java.

In Oct, 2020, I was looking for a solution about running JavaScript code in JVM with identical behavior as in a web browser. V8 seemed to be only candidate and the available solution was J2V8.

However, J2V8 has dropped support for Linux in 2017 and Windows in 2016. The V8 doesn’t even completely support ES6.

In Jan, 2021, I started an open-source project https://github.com/caoccao/Javet/ from scratch. Now, it is fully functional and reaches production level.

Major Features

• 🐧Linux + 🖥️Windows
• Node.js v14.16.1 + V8 v9.0.257
• Dynamic switch between Node.js and V8
• Exposure of the majority of V8 API in JVM
• JS function interception
• Native BigInt and Date
• Javet engine pool
• Easy spring integration
• Live debug with Chrome DevTools

Quick Start

Dependency

Maven

<dependency>
    <groupId>com.caoccao.javet</groupId>
    <artifactId>javet</artifactId>
    <version>0.8.7</version>
</dependency>

Gradle Kotlin DSL

implementation("com.caoccao.javet:javet:0.8.7")

Gradle Groovy DSL

implementation 'com.caoccao.javet:javet:0.8.7'

Hello Javet

// Node.js Mode
try (V8Runtime v8Runtime = V8Host.getNodeInstance().createV8Runtime()) {
    System.out.println(v8Runtime.getExecutor("'Hello Javet'").executeString());
}// V8 Mode
try (V8Runtime v8Runtime = V8Host.getV8Instance().createV8Runtime()) {
    System.out.println(v8Runtime.getExecutor("'Hello Javet'").executeString());
}

Javet v0.8.7 is released.

 

Javet is Java + V8 (JAVa + V + EighT). It is an awesome way of embedding Node.js and V8 in Java.

For more detail, please visit https://github.com/caoccao/Javet

Fix: Intellij IDEA BindException Address already in use

 Have you even met Address already in use on Windows?

The fix is very simple with the following 2 commands.

net stop winnat
net start winnat