Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/update library #26

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,7 @@ trim_trailing_whitespace = false
[*.py]
indent_style = space
indent_size = 4

[*.rb]
indent_style = space
indent_size = 4
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
*.pyc
*.swp
.coverage
*.gem
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,27 @@ wordfilter.addWords(['zebra','elephant'])
wordfilter.blacklisted('this string has zebra in it') # True
```

Or with Ruby:
Install with `gem install wordfilter`.

[![Gem Version](https://badge.fury.io/rb/wordfilter.svg)](https://badge.fury.io/rb/wordfilter)

```ruby
require 'wordfilter'

Wordfilter::blacklisted? "does this string have a bad word in it?" # false

#clear all words from the filter
Wordfilter::clear_list

#add some words to the filter
Wordfilter::add_words(['zebra', 'elephant'])
Wordfilter::blacklisted? "this string has a zebra in it" # true

```

Please note that the Ruby implementation is a global singleton, and is not thread safe.

## Documentation
This is a word filter adapted from code that I use in a lot of my twitter bots. It is based on [a list of words that I've hand-picked](https://github.com/dariusk/wordfilter/blob/master/lib/badwords.json) for exclusion from my bots: essentially, it's a list of things that I would not say myself. Generally speaking, they are "words of oppression", aka racist/sexist/ableist things that I would not say.

Expand Down
5 changes: 4 additions & 1 deletion Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,7 @@ Rake::TestTask.new do |t|
t.libs << "test"
t.test_files = FileList['test/*.rb']
t.verbose = true
end
end

desc "Run tests"
task :default => :test
80 changes: 45 additions & 35 deletions lib/wordfilter.rb
Original file line number Diff line number Diff line change
@@ -1,38 +1,48 @@
require 'json'

module Wordfilter
BadWordsFileName = File.dirname(__FILE__) + "/badwords.json";
@@blacklist = nil

def self.init_first_time
return if(!@@blacklist.nil?)
self.init
end

def self.init
badwords_file = File.read(BadWordsFileName);
@@blacklist = JSON.parse(badwords_file);
end

def self.blacklisted? string
self.init_first_time

string_to_test = string.downcase
@@blacklist.map!{|badword| badword.downcase}

@@blacklist.each{|word|
return true if string_to_test.include? word
}

return false
end

def self.add_words words
self.init_first_time
@@blacklist += words
end

def self.clear_list
@@blacklist = []
end
end
BadWordsFileName = File.dirname(__FILE__) + "/badwords.json"
@@blacklist = nil

def self.init_first_time
return if(!@@blacklist.nil?)
self.init
end

def self.init
badwords_file = File.read(BadWordsFileName)
@@blacklist = JSON.parse(badwords_file)
@@blacklist.map!(&:downcase)
return
end

def self.blacklisted? string
self.init_first_time

string_to_test = string.downcase

@@blacklist.each{|word|
return true if string_to_test.include? word
}

return false
end

def self.add_words words
self.init_first_time
words.map!(&:downcase)
@@blacklist += words
return
end

def self.remove_word word
self.init_first_time
@@blacklist.delete word.downcase
return
end

def self.clear_list
@@blacklist = []
return
end
end
64 changes: 36 additions & 28 deletions test/wordfilter_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,31 +5,39 @@

class WordfilterTest < Test::Unit::TestCase

def setup
super
Wordfilter.init
end

def test_detects_bad_words_in_a_string
assert(Wordfilter::blacklisted?("this string contains the word skank"), "Failed to mark a bad string as bad.");
assert(Wordfilter::blacklisted?("this string contains the word SkAnK"), "Failed to mark a bad string as bad.");
refute(Wordfilter::blacklisted?("this string was clean!"), "Failed to allow a non-bad string.");
end

def test_add_word_to_blacklist
Wordfilter::add_words(['clean']);
assert(Wordfilter::blacklisted?("this string was clean!"), "Failed to blacklist a string containing a new bad word.");
end

def test_added_words_checked_case_insensitively
Wordfilter::add_words(['CLEAN']);
assert(Wordfilter::blacklisted?("this string was clean!"), "Failed to blacklist a string containing a new bad word because of casing differences.");
end

def test_clear_blacklist
Wordfilter::clear_list
refute(Wordfilter::blacklisted?("this string contains the word skank"), "Cleared word list still blacklisting strings.");
Wordfilter::add_words(['skank']);
assert(Wordfilter::blacklisted?("this string contains the word skank"), "Failed to blacklist a string containing a new bad word.");
end
end
def setup
super
Wordfilter.init
end

def test_detects_bad_words_in_a_string
assert(Wordfilter::blacklisted?("this string contains the word skank"), "skank should be true")
assert(Wordfilter::blacklisted?("this string contains the word SkAnK"), "SkAnK should be true")
assert(Wordfilter::blacklisted?("tthis string contains the wordskank"), "wordskank should be true")
assert(Wordfilter::blacklisted?("this string contains the skankword"), "skankword should be true")
refute(Wordfilter::blacklisted?("this string is clean!"), "should be false")
end

def test_add_word_to_blacklist
Wordfilter::add_words(['clean'])
assert(Wordfilter::blacklisted?("this string was clean!"), "Failed to blacklist a string containing a new bad word.")
end

def test_added_words_checked_case_insensitively
Wordfilter::add_words(['CLEAN'])
assert(Wordfilter::blacklisted?("this string was clean!"), "Failed to blacklist a string containing a new bad word because of casing differences.")
end

def test_clear_blacklist
Wordfilter::clear_list
refute(Wordfilter::blacklisted?("this string contains the word skank"), "Cleared word list still blacklisting strings.")
Wordfilter::add_words(['skank'])
assert(Wordfilter::blacklisted?("this string contains the word skank"), "Failed to blacklist a string containing a new bad word.")
end

def test_remove_single_word_from_blacklist
assert(Wordfilter::blacklisted?("I have a prescription."), "Prescription should be blacklisted.")
Wordfilter::remove_word "crip"
refute(Wordfilter::blacklisted?("I have a prescription."), "Prescription should no longer be blacklisted.")
end
end
16 changes: 16 additions & 0 deletions wordfilter.gemspec
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
Gem::Specification.new do |s|
s.name = "wordfilter"
s.version = "0.2.6"
s.date = "2016-01-30"
s.summary = "A small module meant for use in text generators that lets you filter strings for bad words."
s.description = <<EOF
Wordfilter is a small library that can check a string for blacklisted words.
It comes with a pre-defined list of words, but you can add your own.
EOF
s.authors = ["Darius Kazemi", "Eli Brody"]
s.email = ["[email protected]", "[email protected]"]
s.files = ["lib/wordfilter.rb", "lib/badwords.json"]
s.test_files = ["test/wordfilter_test.rb"]
s.homepage = "https://github.com/dariusk/wordfilter"
s.license = "MIT"
end