diff --git a/CHANGES.txt b/CHANGES.txt index 52adf11..16dc0ae 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,12 +1,16 @@ CHANGES +0.3 + +- Changed threadpools to be daemon by default + 0.2.2 - Made pmap behave like map does when a function throws an exception 0.2.1 -- Made it work with java 1.6 +- Made code work with java 1.6 0.2.0 diff --git a/README.md b/README.md index 6fd6949..eb142c3 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ The claypoole library provides threadpool-based parallel versions of Clojure functions such as `pmap`, `future`, and `for`. Claypoole is available in the Clojars repository. Just use this leiningen -dependency: `[com.climate/claypoole "0.2.2"]`. +dependency: `[com.climate/claypoole "0.3"]`. ## Why do you use claypoole? @@ -156,6 +156,11 @@ below.) ## How do I dispose of my threadpools? +*As of Claypoole version 0.3, by default all threadpools are daemon +threadpools, so they should shut down when your main thread exits. But it's +still good practice to clean up these resources. Neither the OS nor the JVM +will take care of them for you!* + The JVM does not automatically garbage collect threads for you. Instead, when you're done with your threadpool, you should use `shutdown` to gently shut down the pool, or `shutdown!` to kill the threads forcibly. @@ -172,10 +177,11 @@ Of course, we have provided a convenience macro `with-shutdown!` that will ``` Alternately, daemon threads will be automatically killed when the JVM process -exits. You can create a daemon threadpool via: +exits. *By default all threadpools are daemon threadpools and will exit when +the main thread exits!* You can create a non-daemon threadpool via: ```clojure -(def pool (cp/threadpool 10 :daemon true)) +(def pool (cp/threadpool 10 :daemon false)) ``` ## How do I set threadpool options? @@ -189,7 +195,7 @@ same as the task priority, described below.) ```clojure (def pool (cp/threadpool (cp/ncpus) - :daemon true + :daemon false :thread-priority 3 :name "my-pool")) ``` @@ -251,6 +257,43 @@ place of a threadpool, and you can use a threadpool just as you would an `ExecutorService`. This means you can create custom threadpools and use them easily. +## OMG My program isn't exiting! + +There are a few cases where threadpools will stop your program from exiting +that can surprise users. We have endeavored to minimize them, but they can +still be problems. + +### My program doesn't exit until 60 seconds after main exits. + +Claypoole actually uses some `clojure.core/future`s. Unfortunately, those +threads are from the agent threadpool, and they are not daemon threads. Those +threads will automatically die 60 seconds after they're used. So if your main +thread doesn't call either `shutdown-agents` or `System/exit`, those threads +will keep going for a while! + +You'll [experience the same +thing](http://tech.puredanger.com/2010/06/08/clojure-agent-thread-pools/) if +you start a `future` or `pmap`--those extra threads have to be explicitly shut +down, or they'll naturally die after 60 seconds. + +The answer is basically to call `(shutdown-agents)` before your main thread +exits. + +_In a future version of Claypoole, we may switch to using a separate daemon +threadpool for those futures. However, that could have other complications, so +we are deferring that decision._ + +### My program doesn't exit ever! + +Claypoole now (as of version 0.3) defaults to daemon threadpools, but before +that threadpools were not daemons. If you have any non-daemon threadpools +running and your main process exits, those threadpools will never die, and your +program will hang indefinitely! + +The answer is: either use daemon threadpools (which is now the default) or be +very careful to shut down your threadpools when you're done with them. See the +above section on disposing of threadpools. + ## Why the name "Claypoole"? The claypoole library is named after [John Claypoole (Betsy Ross's third diff --git a/project.clj b/project.clj index 3311395..2818dcd 100644 --- a/project.clj +++ b/project.clj @@ -12,7 +12,7 @@ ;; and limitations under the License. (defproject com.climate/claypoole - "0.2.2" + "0.3" :description "Claypoole: Threadpool tools for Clojure." :url "http://github.com/TheClimateCorporation/claypoole/" :license {:name "Apache License Version 2.0" diff --git a/src/clj/com/climate/claypoole.clj b/src/clj/com/climate/claypoole.clj index 86a634c..a817e60 100644 --- a/src/clj/com/climate/claypoole.clj +++ b/src/clj/com/climate/claypoole.clj @@ -67,7 +67,7 @@ This takes optional keyword arguments: :daemon, a boolean indicating whether the threads are daemon threads, which will automatically die when the JVM exits, defaults to - false) + true) :name, a string giving the pool name, which will be the prefix of each thread name, resulting in threads named \"name-0\", \"name-1\", etc. Defaults to \"claypoole-[pool-number]\". @@ -82,7 +82,8 @@ ^java.util.concurrent.ScheduledExecutorService ;; NOTE: Although I'm repeating myself, I list all the threadpool-factory ;; arguments explicitly for API clarity. - [n & {:keys [daemon thread-priority] pool-name :name}] + [n & {:keys [daemon thread-priority] pool-name :name + :or {daemon true}}] (impl/threadpool n :daemon daemon :name pool-name diff --git a/src/clj/com/climate/claypoole/impl.clj b/src/clj/com/climate/claypoole/impl.clj index a5c917a..9128d86 100644 --- a/src/clj/com/climate/claypoole/impl.clj +++ b/src/clj/com/climate/claypoole/impl.clj @@ -107,7 +107,8 @@ "Create a ThreadFactory with keyword options including thread daemon status :daemon, the thread name format :name (a string for format with one integer), and a thread priority :thread-priority." - ^ThreadFactory [& {:keys [daemon thread-priority] pool-name :name}] + ^ThreadFactory [& {:keys [daemon thread-priority] pool-name :name + :or {daemon true}}] (let [daemon* (boolean daemon) pool-name* (or pool-name (default-threadpool-name)) thread-priority* (or thread-priority diff --git a/test/com/climate/claypoole_test.clj b/test/com/climate/claypoole_test.clj index add9246..4b04251 100644 --- a/test/com/climate/claypoole_test.clj +++ b/test/com/climate/claypoole_test.clj @@ -592,6 +592,21 @@ (check-shuts-off pmap-like)))) +(deftest test-daemon + (let [daemon? (fn [& args] (.isDaemon (Thread/currentThread)))] + (testing "threadpools are daemon by default" + (cp/with-shutdown! [pool 3] + (is (every? boolean + (cp/pmap pool daemon? (range 100)))))) + (testing "threadpools are daemon by default" + (cp/with-shutdown! [pool (cp/threadpool 3)] + (is (every? boolean + (cp/pmap pool daemon? (range 100)))))) + (testing "we can make non-daemon threadpools" + (cp/with-shutdown! [pool (cp/threadpool 3 :daemon false)] + (is (not-any? boolean + (cp/pmap pool daemon? (range 100)))))))) + (deftest test-future (testing "basic future test" (cp/with-shutdown! [pool 3]