🐛 Update shared value from JS to useFrameProcessor worklet

[jgatto1]

What's happening?

Hey guys! I need to update a shared value from JS to the worklet code running on the useFrameProcessor function but it never updates the code on the worklet side but only on the JS side. How can I properly update the shared value to keep it in sync? I've tried using the useSharedValue from react-native-worklets-core but sometimes crashes somehow and i is also slower. Sorry guys I'm getting too confused working with worklets.
Note: If I add on the useFrameProcessor deep a reactive value like useState it works and I'm able to read the updated value. Just wondering, is it a faster way?

Appreciate any help!

Reproduceable Code

import { useSharedValue } from "react-native-reanimated";

const [update, setUpdate] = useState(false);
const groupOfLinesShared = useSharedValue<number[][] | null>(null);

  useEffect(() => {
    const process = async () => {
      groupOfLinesShared.value = [...groupOfLinesShared.value || [], [2]];
      setUpdate(a => !a);
    };
    process();

    setInterval(() => {
      process();
    }, 3000);
  }, []);

const frameProcessor = useFrameProcessor(
    async (frame) => {
      "worklet";
      console.log('[WORKLET] groupOfLinesShared.value', groupOfLinesShared.value);
      // Here I pretend to get the updated groupOfLinesShared value but the only way is to update the deep of this processor.
      // If I remove the `update` it won't refresh.
}, [update]);

  return (
    <View style={StyleSheet.absoluteFill}>
      <Camera
        style={StyleSheet.absoluteFill}
        video
        device={device}
        ref={camera}
        // isActive={isCameraActive}
        isActive
        frameProcessor={frameProcessor}
        format={compatibleFormat}
        zoom={params.zoom || 1}
        fps={FPS_VALUE}
        outputOrientation="device"
      />
</View>

Relevant log output

LOG  [WORKLET] groupOfLinesShared.value null

Camera Device

{
  "neutralZoom": 1,
  "supportsRawCapture": false,
  "formats": [],
  "supportsLowLightBoost": false,
  "hardwareLevel": "full",
  "minExposure": -8,
  "position": "back",
  "hasFlash": true,
  "minZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "hasTorch": true,
  "maxExposure": 8,
  "sensorOrientation": "portrait",
  "minFocusDistance": 20,
  "maxZoom": 123.75,
  "isMultiCam": false,
  "supportsFocus": true,
  "name": "Back Camera"
}

Device

Iphone 15 Pro

VisionCamera Version

4.7.2

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[maintenance-hans]

Guten Tag, Hans here.

Note

New features, bugfixes, updates and other improvements are all handled mostly by @mrousavy in his free time.
To support @mrousavy, please consider 💖 sponsoring him on GitHub 💖.
Sponsored issues will be prioritized.

[jsg2021]

If you have the latest reanimated, then you have two worklet contexts... because the latest reanimated uses react-native-worklets and this uses the older react-native-worklets-core.

You cannot use the babel plugin for both, so one of them is configured (likely the newer one, because if the older one is configured, reanimated stuff would crash). However, the runtime code expected to exist in the worklet context by the babel plugin is not loaded in worklets created by worklet-core; therefore, it will crash when attempting to use any API of reanimated or react-native-worket... You have to use react-native-worklet-core APIs only in your frame processor...BUT, it's still not smooth sailing... since the worklet is compiled with the newer Babel plugin, it's not 100% compatible with the worklet-core runtime either.

I would recommend pinning Expo to a version that pins reanimated and react-native to compatible versions.

If you want to roll the dice, you can use useSharedValue from worklets-core ... and just poll for changes to the .value of the SharedValue... if you need to write... I'd tear down the processor and start a new one with the updated parameters.

OR... wait for v5, which should solve this.

[rachellauyui]

Any ideas when V5 be ready? The useSkiaFrameProcessor already broken down, and no real way to draw data out.... I run out of idea on how to draw something on top of the frameprocessor.

[ralph1233]

You can do this natively by taking the camera frame from the onFrame function and rendering it using Metal on iOS or OpenGL/Vulkan on Android. Vision Camera already provides an onFrame function.

Any ideas when V5 be ready? The useSkiaFrameProcessor already broken down, and no real way to draw data out.... I run out of idea on how to draw something on top of the frameprocessor.