Implement cursor metadata in screencast #2140
Conversation
YaLTeR
left a comment
YaLTeR
left a comment
There was a problem hiding this comment.
Please rebase onto main, it has the necessary spa_buffer access.
src/pw_utils.rs
Outdated
| // TODO: Can the bitmap be initialized previously in start_cast? | ||
| // Alternativelly, initialize on first pass. For this, bitmap_offset must be 0 on |
There was a problem hiding this comment.
Since the cursor changes at runtime, it should be tracked separately with its own OutputDamageTracker, sent as a spa_meta_bitmap whenever it changes, and otherwise don't send a new bitmap.
There was a problem hiding this comment.
For some reason (pipewire acting up?) if I do that sometimes it does not update the cursor when it changes on the screencast, despite it being sent. For now I'll leave that optimization out and try to look into it later.
I did rebase to main yesterday, but it now seems to have some issue with the cursor bitmap buffer which I haven't figured out yet. Probably some bad pointer offset calculation. I should take a look at it back later today. |
|
For some reason, without 15b4acc#diff-a517890ab4c703e485977a1943ceb154a6bcb3cc4d83135097a2841d569e7326L930-L931 the cast just stops after a second: I've printed the stride and offset from the dmabuf and the current ones on the chunks, and for some reason the chunk ones overflow: They only overflow when I write to the bitmap array. I don't see anything wrong with the bitmap, so maybe something is failing to take the extra bitmap size into account? |
|
Hmm, idk, I looked through this PR's diff and the code in mutter and couldn't see a problem off the bat. Try filling the bitmap with sentinel values (like 0x01, 0x02, 0x03 and so on) to see where it overwrites subsequent data? Maybe check with gdb? |
abmantis
force-pushed
the
window_cast_pointer_meta_on_pw_explicit_sync
branch
from
ad73a89 to
1e654e0
Compare
I'm struggling to find the cause of this. The cursor_meta pointer is slightly different on every frame, which makes it so that I can't just watch that memory space for changes. |
…r_meta_on_pw_explicit_sync
|
@YaLTeR looking at https://docs.pipewire.org/video-src_8c-example.html#a57 the example sets the stride/offset on every frame. Are you sure that is not required? |
|
I'm logging the Notice that the address pairs pointed by the green (frame 1) and blue (frame 2) arrows match, which means that the bitmap_data is in the proper memory space since the data chunk comes right after. |
|
From Matrix:
I'm open to changing it back to set stride/offset every frame, but if it's indeed a bug in PW, it'd be good to get it found and fixed |
|
Non-dmabuf stride and offset do not fall under what I described above of course |
|
Opened an issue on pipewire: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/4884 |
|
Thanks, subscribed |
|
I was able to confirm that the PR linked to that issue fixes the problem! 🥳 |
|
Awesome! I guess you should add a runtime check for pipewire version with the fix though. |
…r_meta_on_pw_explicit_sync
| ); | ||
| } | ||
|
|
||
| let fd = (*(*spa_buffer).datas).fd; |
There was a problem hiding this comment.
we should now be able to also optimize the screencast code to not render a new frame when there is no damage to it, and send only cursor metadata. But that appears to require some extra refactoring, so I suggest leaving it for a follow-up PR instead.
|
Is there more work that needs to be done on this for cursor data to be sent for screencasting? I have built with these changes locally but the cursor still does not appear in screencasts. Would love an update on this. Thanks. |
|
fwiw I haven't had a chance to look at this again yet; I'll try to before the next release if I find time. Testing is very welcome |
No, it should be done. What version of pipewire do you have installed? |
I am on 1.2.7 which is what comes with Ubuntu 25.04. Will I need to build pipewire from source? |
Yeah, you need |
Updated to pipewire 1.4.7 and the cursor still does not appear in screencasts. In the journal it says that |
|
After compiling pipewire 1.5.8 from source, I get the following in journal:
How do I further troubleshoot this? Have you tested on later version of pipewire or only 1.4.8? |
That log message is a bit misleading since there is no default bitmap. I've just updated. Anyway, can you enable debug logs and see if something else pops up? I don't know why the rendering of the cursor elements is failing. |
I don't know if these logs help but it's all I get in journal which is still flooded with these "error rendering cursor" messages. |
|
Along with the cursor issue, niri crashed completely when I started a screencast: Will update when I have reproduced it with verbose logs. I never experienced this prior to building with these changes. |
|
Was able to reproduce consistently by starting a screencast and then dragging the screencasted window via Mod+MB1. @abmantis are you able to reproduce this on your system? |
I've just tried that and cannot reproduce the crash. But I'll look at the unwraps and see if I can make them safer. |
…r_meta_on_pw_explicit_sync
|
@ASong5 can you try again and check the logs? The |
|
I just tested and the crashes are gone! However, cursor data is still not being sent properly for me: #511 is the most related issue I can find, but it seems to be fixed already and mainly affected nvidia cards whereas I use amd: I'm not sure if you had a chance to test with newer versions of pipewire than 1.4.8 but I will downgrade to 1.4.8 and report back in case it is an issue with newer versions of pipewire. |
|
I've just tested it with pipewire 1.4.9 since that is now available on NixOS. |
|
Tested with pipewire version 1.4.8 and the issue persists. I'm running niri via SDDM |
|
Same here. It also breaks the original cursor capture function. The error occurred at edited:I found that But it's not the reason why the cursor disappears. |
4 participants
This implements cursor metadata for pipewire window and output screencasts.
Fixes #115.