[iOS] Distinguish between mouse and stylus when hovering#3991
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request fixes incorrect pointer type reporting in iOS hover gestures by distinguishing between mouse and stylus input. The fix addresses issue #3977 where hover gestures were reporting incorrect pointer types due to using the wrong enum constant and inability to distinguish between mouse and stylus.
Changes:
- Fixed pointer type enum usage from
UITouchTypePenciltoRNGestureHandlerStylus - Added logic to detect mouse vs stylus using
zOffsetproperty (iOS 16.1+) - Changed method signature parameter type from
UIGestureRecognizertoUIHoverGestureRecognizerfor type safety
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Looks good! Though I think the logic should be inverted and assume default as mouse for ios<16.1. Due to the bug where we returned |
bb3c39d to
a1fbe44
Compare
Sounds good to me. Flipped it around. |
m-bert
left a comment
There was a problem hiding this comment.
Thank you for this PR! I’ve left some comments – feel free to ask if anything needs further clarification or guidance. 😅
packages/react-native-gesture-handler/apple/Handlers/RNHoverHandler.m
Outdated
Show resolved
Hide resolved
| - (RNGestureHandlerEventExtraData *)eventExtraData:(UIGestureRecognizer *)recognizer | ||
| - (RNGestureHandlerEventExtraData *)eventExtraData:(UIHoverGestureRecognizer *)recognizer | ||
| { | ||
| RNGestureHandlerPointerType pointerType = RNGestureHandlerMouse; |
There was a problem hiding this comment.
Handlers already have pointerType, along with setCurrentPointerType. Now, I know that it accepts event, but maybe if we can't use it we could change pointerType somewhere inside recognizer and reset it when gesture ends?
There was a problem hiding this comment.
That's a good point and it would make sense to try to conform. Can't do it exactly the same way as some of the other gestures due to the hover one being a bit special it seems.
I tried with overriding the handleGesture method in RNHoverGestureHandler to be able to detect and set the pointer type when the gesture begins. And as you said, the setCurrentPointerType method expects an UIEvent (with touches available), which we do not have here. So instead we set the _pointerType directly.
- (void)handleGesture:(UIHoverGestureRecognizer *)recognizer
{
if (recognizer.state == UIGestureRecognizerStateBegan) {
_pointerType = [self detectPointerType:recognizer];
}
[super handleGesture:recognizer];
}The detection method is the same as before;
- (RNGestureHandlerPointerType)detectPointerType:(UIHoverGestureRecognizer *)recognizer
{
RNGestureHandlerPointerType pointerType = RNGestureHandlerMouse;
#if CHECK_TARGET(16_1)
if (@available(iOS 16.1, *)) {
if (recognizer.zOffset > 0.0) {
pointerType = RNGestureHandlerStylus;
}
}
#endif
return pointerType;
}And then obviously in the eventExtraData method we only need to update the argument to send _pointerType.
// ...
withPointerType:_pointerType];
// ...I will test a bit more during the day when I have some time.
Any feedback on this?
There was a problem hiding this comment.
tl;dr
We could use handleGesture along with setCurrentPointerType. If you're willing to do so, you can find details below. If you don't have time (or something else) let me know, I'll introduce necessary changes into this PR 😅
handleGesture
I tried with overriding the
handleGesturemethod inRNHoverGestureHandlerto be able to detect and set the pointer type when the gesture begins.
In that case you'd have to change target when initializing recognizer:
- if ((self = [super initWithTarget:_gestureHandler action:@selector(handleGesture:)])) {
+ if ((self = [super initWithTarget:self action:@selector(handleGesture:)])) {Also if you decide to handle it in handleGesture it would be nice to check recognizer's state to only set it once, when gesture begins.
if(self.state == UIGestureRecognizerStateBegan){
[_gestureHandler setCurrentPointerType:nil];
}And remember to call this method on _gestureHandler:
[_gestureHandler handleGesture:self];setCurrentPointerType
The other thing is, _pointerType is not easily accessible from recognizer. What we could do is to handle this inside already existing setCurrentPointerType. Since we do not need UIEvent, we can make it nullable. Also handlers' have access to their recognizers, so we could do something like this:
if([self.recognizer isKindOfClass:[UIHoverGestureRecognizer class]]) {
// _pointerType = ...
return;
}and for safety we could assume default value if for some reason we enter this function from other recognizer without event:
if(event == nil){
_pointerType = RNGestureHandlerTouch;
return;
}There was a problem hiding this comment.
I pushed a new commit with this approach in mind.
In that case you'd have to change target when initializing recognizer:
I don't think this is the case, self would be the recognizer here, right? The handleGesture method is on the handler. Which is where i've added an override for handleGesture, setting the pointerType (now via setCurrentPointerType instead of directly). Well, at least it works, maybe I'm missing another detail here?
But I must say it feels a bit off to have hover specific logic in the generic GestureHandler instead of inside the HoverGesture file. Welcoming any refactoring suggestions.
@m-bert Feel free to make further modifications if you have any good ideas, and I'll test it on my devices.
There was a problem hiding this comment.
I don't think this is the case, self would be the recognizer here, right?
True, I though about adding handleGesture to recognizer like we do in other handlers 😅 I've done it in 4c0271e.
But I must say it feels a bit off to have hover specific logic in the generic GestureHandler instead of inside the HoverGesture file. Welcoming any refactoring suggestions.
I agree. I've moved it to Hover in 1b69398.
It would be great if you could confirm that it works on your end 😅
There was a problem hiding this comment.
Seems fine to me!
Tested with a apple pencil pro, connected mouse, simulated mouse and touch. All reporting the expected pointer type :)
Do you need anything else from me at this stage?
There was a problem hiding this comment.
Glad to hear that!
Do you need anything else from me at this stage?
No, thank you very much for your time ❤️
There was a problem hiding this comment.
Or actually, there might be. When I do some changes based on reviews, I'll ask you again to test before merging if we didn't break anything. Is that ok? 😅
There was a problem hiding this comment.
Could you please check it one more time @AndreasHogstrom? I think now we are done with changes 😅
There was a problem hiding this comment.
Works perfectly fine on all my devices and hover-capable peripherals. So it at least covers my use cases :)
Thanks a lot for input and changes 👍
a1fbe44 to
a249ef9
Compare
| - (void)setCurrentPointerType:(UIEvent *)event | ||
| { | ||
| if (event == nil) { | ||
| _pointerType = RNGestureHandlerTouch; | ||
| return; | ||
| } |
There was a problem hiding this comment.
This is a bit misleading. I'd expect setCurrentPointerType to accept RNGestureHandlerPointerType, and another method - setCurrentPointerTypeForEvent that would accept an event.
Then the one accepting event can stay as nonnull, and call the other one. The hover gesture can then set whatever pointer type it needs directly.
| - (void)handleGesture:(UIHoverGestureRecognizer *)recognizer | ||
| { | ||
| if (recognizer.state == UIGestureRecognizerStateBegan) { | ||
| [_gestureHandler setCurrentPointerType:nil]; |
There was a problem hiding this comment.
Works, but I've changed default pointer type to mouse: b102e96
Description
Improves the hover gesture on iOS by distinguishing between a mouse and a stylus, if possible.
With these changes the handler will now assume a mouse (just like previously with the enum bug), but if
zOffsetis available (iOS 16.1) and it is over 0.0 it will instead be recognised as styus.Maybe it makes more sense to invert the logic and assume a mouse as default, and stylus if > 0. Let me know what you think.See #3977 for more context.
Fixes #3977
Test plan
Here's a small test page that can be added to the basic-example app for testing;
Mouse is testable in the iPad simulator by enabling
I/O -> Input -> Send Pointer to Devicein the top menus.Mouse is also testable on a real iPad by Linking keyboard and mouse to it from a mac in the Displays settings.
Apple Pencil is only testable with an actual pencil on a real iPad.