Custom emoji with non-ASCII names don’t render #533

Open
opened 2024-03-17 16:45:04 +01:00 by Tirifto · 6 comments

Hello! It appears that an instance’s custom emoji will not render in Iceshrimp if the name (shortcode) contains multi-byte characters. Consider for example this post as seen from Pleroma:

Screenshot of the post on Pleroma, showing a selection of custom emoji.

And as seen from an Iceshrimp instance:

Screenshot of the post on Iceshrimp, not showing the emoji themselves, but rather just their names inside pairs of colons. The names contain non-ASCII characters used in foreign alphabets and writing systems.

This should be supported (for alphanumeric characters) so that instances can name their emoji in whichever language, and so that languages whose writing system doesn’t mostly fit in ASCII (like English does) are not being disadvantaged in the Fediverse.

Hello! It appears that an instance’s custom emoji will not render in Iceshrimp if the name (shortcode) contains multi-byte characters. Consider for example [this post](https://jam.xwx.moe/notice/AeTkKX1MfKhtjPUun2) as seen from Pleroma: ![Screenshot of the post on Pleroma, showing a selection of custom emoji.](/attachments/c8d8b578-0951-4c0a-96c4-cc483a8bb919) And as seen from an [Iceshrimp instance](https://snug.moe/notes/9p8hm8itn2sfzuyc): ![Screenshot of the post on Iceshrimp, not showing the emoji themselves, but rather just their names inside pairs of colons. The names contain non-ASCII characters used in foreign alphabets and writing systems.](/attachments/3b8946eb-49b7-4f36-b052-480fd6a21b27) This should be supported (for alphanumeric characters) so that instances can name their emoji in whichever language, and so that languages whose writing system doesn’t mostly fit in ASCII (like English does) are not being disadvantaged in the Fediverse.
Owner

This is an mfm.js issue (https://github.com/misskey-dev/mfm.js/blob/develop/src/internal/parser.ts#L625-L635), we'd have to fork that dependency to fix this issue.

This is an mfm.js issue (https://github.com/misskey-dev/mfm.js/blob/develop/src/internal/parser.ts#L625-L635), we'd have to fork that dependency to fix this issue.
Owner

I think a more reasonable regex would be ^[^.@\s]+$ (@ is invalid as it separates localpart & host for remote emoji, . because I highly doubt other software would handle that correctly and \s for whitespace)

I think a more reasonable regex would be `^[^.@\s]+$` (`@` is invalid as it separates localpart & host for remote emoji, `.` because I highly doubt other software would handle that correctly and `\s` for whitespace)
Owner

I've updated the regex in Iceshrimp.NET to the one mentioned in the comment above (f5aa7f6ff6)

I've updated the regex in Iceshrimp.NET to the one mentioned in the comment above (https://iceshrimp.dev/iceshrimp/Iceshrimp.NET/commit/f5aa7f6ff6927790900ab3afb04ffc59ae65afdd)
Author

Thank you! ^ ^

An issue has been opened for Misskey (so far without dev reaction); could it be more appropriate to submit a report or a pull request to mfm.js directly, and would fixing the issue there transitively fix the issue in Iceshrimp, too?

Thank you! ^ ^ An issue has been opened [for Misskey](https://github.com/misskey-dev/misskey/issues/13439) (so far without dev reaction); could it be more appropriate to submit a report or a pull request to `mfm.js` directly, and would fixing the issue there transitively fix the issue in Iceshrimp, too?
Member

It would not, differences between our version and theirs are probably so big that it's not that easy to deal with most likely

It would not, differences between our version and theirs are probably so big that it's not that easy to deal with most likely
Owner

I just checked & there is no functional difference between the latest mfm.js version and the one we currently link against. Should upstream fix this, we can pull in the changes without issues.

I just checked & there is no functional difference between the latest mfm.js version and the one we currently link against. Should upstream fix this, we can pull in the changes without issues.
Sign in to join this conversation.
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: iceshrimp/iceshrimp#533
No description provided.