Undefined Symbols in V8 Monolith

There are many applications and libraries acting as embedders to Google V8 JavaScript engine. My open source project Javet is one of them. When I was about to upgrade to V8 v11.7, the build was broken. I'd like to share the troubleshooting story in this post with you.

Undefined symbols absl::time_internal::cctz::local_time_zone()

The Javet build on V8 v11.7 worked well on Windows, Linux and Android, but failed on Mac x86_64 and arm64 with the following cmake error logs.

Undefined symbols for architecture x86_64:
  "_CFRelease", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
  "_CFStringGetCString", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
  "_CFStringGetLength", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
  "_CFStringGetMaximumSizeForEncoding", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
  "_CFTimeZoneCopyDefault", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
  "_CFTimeZoneGetName", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
ld: symbol(s) not found for architecture x86_64

Undefined symbols for architecture arm64:
  "_CFRelease", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
  "_CFStringGetCString", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
  "_CFStringGetLength", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
  "_CFStringGetMaximumSizeForEncoding", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
  "_CFTimeZoneCopyDefault", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
  "_CFTimeZoneGetName", referenced from:
      absl::time_internal::cctz::local_time_zone() in libv8_monolith.a(time_zone_lookup.o)
ld: symbol(s) not found for architecture arm64

Analysis

V8 has a build target called v8_monolith which is for embedders. It usually has all the symbols built in libv8_monolith.a. However it forgets to include Abseil - C++ Common Libraries as the error log shows undefined symbols during the link phase.

So, the goal is to find or build those missing symbols for the linker to work properly.

Solution

After going over the official Abseil doc, I found the solution at Abseil CMake Build Instructions. Basically, I needed to add Abseil to the Javet CMakeList.txt so that the linker won't complain anymore.

add_subdirectory(${V8_DIR}/third_party/abseil-cpp ${V8_RELEASE_DIR}/third_party/abseil-cpp)
target_link_libraries(Javet PUBLIC absl::base absl::time)

• ${V8_DIR} is the directory of the V8 source code.
• ${V8_RELEASE_DIR} is the directory of the build target.
  • x86_64: ${V8_DIR}/out.gn/x64.release
  • arm64: ${V8_DIR}/out.gn/arm64.release

After this patch was applied, the build and test passed.

CAPEA_KPEAVIsolate or CUPEA_KPEAVIsolate?

I tried to build V8 monolith on Windows 10 for v11.4 and v11.5 recently. The builds were good. However, when I tried to build my app linking v8_monolith.lib, I got the following error messages.

javet_converter.obj : error LNK2019: unresolved external symbol "private: static unsigned __int64 * __cdecl v8::internal::HandleScope::Extend(class v8::internal::Isolate *)" (?Extend @HandleScope@internal@v8@@CAPEA_KPEAVIsolate@23@@Z) referenced in function "class _jobject * __cdecl Javet::Converter::ToExternalV8Value(struct JNIEnv_ *,class Javet::V8Runtime const  *,class v8::Local<class v8::Context> const &,class v8::internal::Object const &)" (?ToExternalV8Value@Converter@Javet@@YAPEAV_jobject@@PEAUJNIEnv_@@PEBVV8Runtime@2@AEBV?$Local@VCont ext@v8@@@v8@@AEBVObject@internal@7@@Z)

I located the symbols via dumpbin and found the highlighted one actually is CUPEA_KPEAVIsolate.

I'm not sure what caused the corrupted V8 build. I fixed it by binary searching and replacing CUPEA_KPEAVIsolate with CAPEA_KPEAVIsolate, and it works.

Serialize and Deserialize PageImpl in Jackson

It's very common to put paginated query results in Redis via Jackson serialization and deserialization. However, org.springframework.data.domain.PageImpl doesn't expose a default constructor. Even worse, that applies to org.springframework.data.domain.Sort and org.springframework.data.domain.Sort.Order as well. There are a couple of workarounds. I'd like to introduce a less intrusive approach for your reference.

The idea is to write a Jackson module which injects a custom serializer and deserializer for org.springframework.data.domain.PageImpl. Let's see how to do that.

1. Define an interface.

public interface IJPADataPage {
    String _CLASS = "@class";
    String CONTENT = "content";
    String DIRECTION = "direction";
    String IGNORE_CASE = "ignoreCase";
    String NULL_HANDLING = "nullHandling";
    String NUMBER = "number";
    String PROPERTY = "property";
    String SIZE = "size";
    String SORT = "sort";
    String TOTAL_ELEMENTS = "totalElements";
}

2. Define a Serializer.

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.jsontype.TypeSerializer;
import com.fasterxml.jackson.databind.ser.std.StdSerializer;
import org.springframework.data.domain.PageImpl;
import org.springframework.data.domain.Sort;

import java.io.IOException;

@SuppressWarnings("unchecked")
public class JPADataPageSerializer
        extends StdSerializer<PageImpl>
        implements IJPADataPage {
    public JPADataPageSerializer(Class<PageImpl> t) {
        super(t);
    }

    public JPADataPageSerializer() {
        this(null);
    }

    @Override
    public void serialize(
            PageImpl value,
            JsonGenerator jsonGenerator,
            SerializerProvider serializerProvider) throws IOException {
        jsonGenerator.writeStartObject();
        jsonGenerator.writeStringField(_CLASS, PageImpl.class.getName());
        jsonGenerator.writePOJOField(CONTENT, value.getContent());
        jsonGenerator.writeNumberField(NUMBER, value.getNumber());
        jsonGenerator.writeNumberField(SIZE, value.getSize());
        jsonGenerator.writeNumberField(TOTAL_ELEMENTS, value.getTotalElements());
        jsonGenerator.writeArrayFieldStart(SORT);
        if (!value.getSort().isEmpty()) {
            for (Sort.Order order : value.getSort()) {
                jsonGenerator.writeStartObject();
                jsonGenerator.writeStringField(PROPERTY, order.getProperty());
                jsonGenerator.writeStringField(DIRECTION, order.getDirection().name());
                jsonGenerator.writeBooleanField(IGNORE_CASE, order.isIgnoreCase());
                jsonGenerator.writeStringField(NULL_HANDLING, order.getNullHandling().name());
                jsonGenerator.writeEndObject();
            }
        }
        jsonGenerator.writeEndArray();
        jsonGenerator.writeEndObject();
    }

    @Override
    public void serializeWithType(
            PageImpl value,
            JsonGenerator jsonGenerator,
            SerializerProvider serializerProvider,
            TypeSerializer typeSer) throws IOException {
        serialize(value, jsonGenerator, serializerProvider);
    }
}

3. Define a Deserializer.

import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.deser.std.StdDeserializer;
import com.fasterxml.jackson.databind.jsontype.TypeDeserializer;
import com.fasterxml.jackson.databind.node.ArrayNode;
import org.springframework.data.domain.PageImpl;
import org.springframework.data.domain.PageRequest;
import org.springframework.data.domain.Sort;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@SuppressWarnings("unchecked")
public class JPADataPageDeserializer
        extends StdDeserializer<PageImpl>
        implements IJPADataPage {
    public JPADataPageDeserializer(Class<?> type) {
        super(type);
    }

    public JPADataPageDeserializer() {
        this(null);
    }

    @Override
    public PageImpl deserialize(
            JsonParser jsonParser,
            DeserializationContext deserializationContext) throws IOException {
        JsonNode jsonNode = jsonParser.getCodec().readTree(jsonParser);
        List<?> content = deserializationContext.readTreeAsValue(jsonNode.get(CONTENT), List.class);
        int number = jsonNode.get(NUMBER).asInt(0);
        int size = jsonNode.get(SIZE).asInt(1);
        int totalElements = jsonNode.get(TOTAL_ELEMENTS).asInt(0);
        List<Sort.Order> orders = new ArrayList<>();
        ArrayNode arrayNode = (ArrayNode) jsonNode.get(SORT);
        if (!arrayNode.isEmpty()) {
            for (JsonNode jsonNodeOrder : arrayNode) {
                String property = jsonNodeOrder.get(PROPERTY).asText();
                Sort.Direction direction = Sort.Direction.valueOf(jsonNodeOrder.get(DIRECTION).asText());
                boolean ignoreCase = jsonNodeOrder.get(IGNORE_CASE).asBoolean();
                Sort.NullHandling nullHandling = Sort.NullHandling.valueOf(jsonNodeOrder.get(NULL_HANDLING).asText());
                Sort.Order order = new Sort.Order(direction, property, nullHandling);
                if (ignoreCase) {
                    order = order.ignoreCase();
                }
                orders.add(order);
            }
        }
        PageRequest pageRequest = PageRequest.of(number, size, Sort.by(orders));
        return new PageImpl(content, pageRequest, totalElements);
    }

    @Override
    public Object deserializeWithType(
            JsonParser jsonParser,
            DeserializationContext deserializationContext,
            TypeDeserializer typeDeserializer) throws IOException {
        return deserialize(jsonParser, deserializationContext);
    }
}

4. Define a module.

import com.fasterxml.jackson.core.Version;
import com.fasterxml.jackson.databind.module.SimpleModule;
import org.springframework.data.domain.PageImpl;

public class JPADataModule extends SimpleModule {
    public static final String NAME = "JPA Data Module";
    public static final Version VERSION = new Version(0, 1, 0, null, null, null);

    public JPADataModule() {
        super(NAME, VERSION);
        addSerializer(PageImpl.class, new JPADataPageSerializer());
        addDeserializer(PageImpl.class, new JPADataPageDeserializer());
    }
}

5. Register the module.

objectMapper.registerModule(new JPADataModule());

Now Jackson is able to handle PageImpl and the Redis cache works.

IDEA / HandBrake / WSL Port Conflict

Recently I upgraded HandBrake to the latest version, however, it stopped working. The root cause is TCP port conflict. As usual, I ran net stop winnat && net start winnat and it worked. But, that broke WSL network.

I didn't want to reset the winnat every time I started Windows with a broken WSL. Finally, I found the root cause: Hyper-V reserves huge amount of TCP ports after Windows is up. The fix is to tell Hyper-V to avoid those commonly used TCP ports.

The following command can show you the excluded TCP port ranges.

> netsh int ipv4 show excludedportrange protocol=tcp

Protocol tcp Port Exclusion Ranges

Start Port    End Port
----------    --------
        80          80
      1000        1010
      1020        1030
      ......

As you can see, huge amount of TCP ports are reserved by Hyper-V. So, let's tell Hyper-V not to be that greedy by executing the following command.

> netsh int ipv4 set dynamic tcp start=49152 num=16384

That command tells Windows to allow dynamic TCP ports from 49152 so that Hyper-V is only able to reserve TCP ports starting from 49152. The following command can verify the setting is correct.

> netsh int ipv4 show dynamic protocol=tcp

Protocol tcp Dynamic Port Range
---------------------------------
Start Port      : 49152
Number of Ports : 16384

Once the new dynamic TCP port range is set, just reboot your machine and everything goes back to normal. IDEA can start smoothly, WSL network is always on, HandBrake works all the time.

dlopen failed: cannot locate symbol "__aarch64_ldadd4_relax"

In Android development, it's rare to meet the following error.

dlopen failed: cannot locate symbol "__aarch64_ldadd4_relax"

I searched the whole internet for a solution, but couldn't get a practical one. Actually, the root cause is simple: The Android NDK is too old.

The solution in my case is:

1. Upgrade CMake to the latest version.
2. Upgrade Android NDK to the latest version.

Javet Supports Android ABI >= 21

 Starting from v1.0.3, Javet supports Android ABI >= 21. Enjoy!

Javet for Android is Released

Javet is Java + V8 (JAVa + V + EighT). It is an awesome way of embedding Node.js and V8 in Java.

It's been more than half a year for the Javet users to wait for the Android support. Now, Javet has officially supported Android.

The API and coding experience are identical to the ones on Linux, Mac OS and Windows.