FreakU
/
mastodon
Archived
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
This repository has been archived on 2024-07-22. You can view files and clone it, but cannot push or open issues or pull requests.
mastodon/spec/models/glitch/keyword_mute_spec.rb

90 lines
2.8 KiB
Ruby
Raw Normal View History

Add KeywordMute model. Gist of the proposed keyword mute implementation: Keyword mutes are represented server-side as one keyword per record. For each account, there exists a keyword regex that is generated as one big alternation of all keywords. This regex is cached (in Redis, I guess) so we can quickly get it when filtering in FeedManager.
2017-10-09 15:28:28 -07:00
require 'rails_helper'
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
RSpec.describe Glitch::KeywordMute, type: :model do
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
let(:alice) { Fabricate(:account, username: 'alice').tap(&:save!) }
let(:bob) { Fabricate(:account, username: 'bob').tap(&:save!) }
Spec out KeywordMute interface. #164.
2017-10-14 00:28:20 -07:00
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
describe '.matcher_for' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
let(:matcher) { Glitch::KeywordMute.matcher_for(alice) }
Spec out KeywordMute interface. #164.
2017-10-14 00:28:20 -07:00
Don't add \b to whole-word keywords that don't start with word characters. Ditto for ending with \b. Consider muting the phrase "(hot take)". I stipulate it is reasonable to enter this with the default "match whole word" behavior. Under the old behavior, this would be encoded as \b\(hot\ take\)\b However, if \b is before the first character in the string and the first character in the string is not a word character, then the match will fail. Ditto for after. In our example, "(" is not a word character, so this will not match statuses containing "(hot take)", and that's a very surprising behavior. To address this, we only add leading and trailing \b to keywords that start or end with word characters.
2017-10-21 22:24:32 -07:00
describe 'with no mutes' do
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
before do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.delete_all
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
end
it 'does not match' do
expect(matcher =~ 'This is a hot take').to be_falsy
end
Spec out KeywordMute interface. #164.
2017-10-14 00:28:20 -07:00
end
Don't add \b to whole-word keywords that don't start with word characters. Ditto for ending with \b. Consider muting the phrase "(hot take)". I stipulate it is reasonable to enter this with the default "match whole word" behavior. Under the old behavior, this would be encoded as \b\(hot\ take\)\b However, if \b is before the first character in the string and the first character in the string is not a word character, then the match will fail. Ditto for after. In our example, "(" is not a word character, so this will not match statuses containing "(hot take)", and that's a very surprising behavior. To address this, we only add leading and trailing \b to keywords that start or end with word characters.
2017-10-21 22:24:32 -07:00
describe 'with mutes' do
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
it 'does not match keywords set by a different account' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.create!(account: bob, keyword: 'take')
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
expect(matcher =~ 'This is a hot take').to be_falsy
end
it 'does not match if no keywords match the status text' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.create!(account: alice, keyword: 'cold')
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
expect(matcher =~ 'This is a hot take').to be_falsy
end
Allow keywords to match either substrings or whole words. Word-boundary matching only works as intended in English and languages that use similar word-breaking characters; it doesn't work so well in (say) Japanese, Chinese, or Thai. It's unacceptable to have a feature that doesn't work as intended for some languages. (Moreso especially considering that it's likely that the largest contingent on the Mastodon bit of the fediverse speaks Japanese.) There are rules specified in Unicode TR29[1] for word-breaking across all languages supported by Unicode, but the rules deliberately do not cover all cases. In fact, TR29 states For example, reliable detection of word boundaries in languages such as Thai, Lao, Chinese, or Japanese requires the use of dictionary lookup, analogous to English hyphenation. So we aren't going to be able to make word detection work with regexes within Mastodon (or glitchsoc). However, for a first pass (even if it's kind of punting) we can allow the user to choose whether they want word or substring detection and warn about the limitations of this implementation in, say, docs. [1]: https://unicode.org/reports/tr29/ https://web.archive.org/web/20171001005125/https://unicode.org/reports/tr29/
2017-10-15 17:49:22 -07:00
it 'considers word boundaries when matching' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.create!(account: alice, keyword: 'bob', whole_word: true)
Allow keywords to match either substrings or whole words. Word-boundary matching only works as intended in English and languages that use similar word-breaking characters; it doesn't work so well in (say) Japanese, Chinese, or Thai. It's unacceptable to have a feature that doesn't work as intended for some languages. (Moreso especially considering that it's likely that the largest contingent on the Mastodon bit of the fediverse speaks Japanese.) There are rules specified in Unicode TR29[1] for word-breaking across all languages supported by Unicode, but the rules deliberately do not cover all cases. In fact, TR29 states For example, reliable detection of word boundaries in languages such as Thai, Lao, Chinese, or Japanese requires the use of dictionary lookup, analogous to English hyphenation. So we aren't going to be able to make word detection work with regexes within Mastodon (or glitchsoc). However, for a first pass (even if it's kind of punting) we can allow the user to choose whether they want word or substring detection and warn about the limitations of this implementation in, say, docs. [1]: https://unicode.org/reports/tr29/ https://web.archive.org/web/20171001005125/https://unicode.org/reports/tr29/
2017-10-15 17:49:22 -07:00
expect(matcher =~ 'bobcats').to be_falsy
end
it 'matches substrings if whole_word is false' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.create!(account: alice, keyword: 'take', whole_word: false)
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
Allow keywords to match either substrings or whole words. Word-boundary matching only works as intended in English and languages that use similar word-breaking characters; it doesn't work so well in (say) Japanese, Chinese, or Thai. It's unacceptable to have a feature that doesn't work as intended for some languages. (Moreso especially considering that it's likely that the largest contingent on the Mastodon bit of the fediverse speaks Japanese.) There are rules specified in Unicode TR29[1] for word-breaking across all languages supported by Unicode, but the rules deliberately do not cover all cases. In fact, TR29 states For example, reliable detection of word boundaries in languages such as Thai, Lao, Chinese, or Japanese requires the use of dictionary lookup, analogous to English hyphenation. So we aren't going to be able to make word detection work with regexes within Mastodon (or glitchsoc). However, for a first pass (even if it's kind of punting) we can allow the user to choose whether they want word or substring detection and warn about the limitations of this implementation in, say, docs. [1]: https://unicode.org/reports/tr29/ https://web.archive.org/web/20171001005125/https://unicode.org/reports/tr29/
2017-10-15 17:49:22 -07:00
expect(matcher =~ 'This is a shiitake mushroom').to be_truthy
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
end
it 'matches keywords at the beginning of the text' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.create!(account: alice, keyword: 'take')
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
expect(matcher =~ 'Take this').to be_truthy
end
it 'matches keywords at the beginning of the text' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.create!(account: alice, keyword: 'take')
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
expect(matcher =~ 'This is a hot take').to be_truthy
end
it 'matches if at least one keyword case-insensitively matches the text' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.create!(account: alice, keyword: 'hot')
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
Fix case-insensitive match scenario; test some word ornamentation. #164.
2017-10-14 18:41:52 -07:00
expect(matcher =~ 'This is a HOT take').to be_truthy
end
it 'matches keywords surrounded by non-alphanumeric ornamentation' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.create!(account: alice, keyword: 'hot')
Fix case-insensitive match scenario; test some word ornamentation. #164.
2017-10-14 18:41:52 -07:00
Don't add \b to whole-word keywords that don't start with word characters. Ditto for ending with \b. Consider muting the phrase "(hot take)". I stipulate it is reasonable to enter this with the default "match whole word" behavior. Under the old behavior, this would be encoded as \b\(hot\ take\)\b However, if \b is before the first character in the string and the first character in the string is not a word character, then the match will fail. Ditto for after. In our example, "(" is not a word character, so this will not match statuses containing "(hot take)", and that's a very surprising behavior. To address this, we only add leading and trailing \b to keywords that start or end with word characters.
2017-10-21 22:24:32 -07:00
expect(matcher =~ '(hot take)').to be_truthy
end
it 'escapes metacharacters in keywords' do
Glitch::KeywordMute.create!(account: alice, keyword: '(hot take)')
expect(matcher =~ '(hot take)').to be_truthy
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
end
it 'uses case-folding rules appropriate for more than just English' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.create!(account: alice, keyword: 'großeltern')
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
expect(matcher =~ 'besuch der grosseltern').to be_truthy
end
it 'matches keywords that are composed of multiple words' do
Move KeywordMute into Glitch namespace. There are two motivations for this: 1. It looks like we're going to add other features that require server-side storage (e.g. user notes). 2. Namespacing glitchsoc modifications is a good idea anyway: even if we do not end up doing (1), if upstream introduces a keyword-mute feature that also uses a "KeywordMute" model, we can avoid some merge conflicts this way and work on the more interesting task of choosing which implementation to use.
2017-10-21 12:47:17 -07:00
Glitch::KeywordMute.create!(account: alice, keyword: 'a shiitake')
Spec out KeywordMute interface. #164.
2017-10-14 00:28:20 -07:00
Rework KeywordMute interface to use a matcher object; spec out matcher. #164. A matcher object that builds a match from KeywordMute data and runs it over text is, in my view, one of the easier ways to write examples for this sort of thing.
2017-10-14 18:36:53 -07:00
expect(matcher =~ 'This is a shiitake').to be_truthy
expect(matcher =~ 'This is shiitake').to_not be_truthy
end
Spec out KeywordMute interface. #164.
2017-10-14 00:28:20 -07:00
end
end
Add KeywordMute model. Gist of the proposed keyword mute implementation: Keyword mutes are represented server-side as one keyword per record. For each account, there exists a keyword regex that is generated as one big alternation of all keywords. This regex is cached (in Redis, I guess) so we can quickly get it when filtering in FeedManager.
2017-10-09 15:28:28 -07:00
end