Merge pull request #6 from TheClimateCorporation/threadpool-cleanup

Make threadpools daemon by default and warn users

CHANGES.txt

@@ -1,12 +1,16 @@
 CHANGES
 
+0.3
+
+- Changed threadpools to be daemon by default
+
 0.2.2
 
 - Made pmap behave like map does when a function throws an exception
 
 0.2.1
 
-- Made it work with java 1.6
+- Made code work with java 1.6
 
 0.2.0
 

README.md

@@ -4,7 +4,7 @@ The claypoole library provides threadpool-based parallel versions of Clojure
 functions such as `pmap`, `future`, and `for`.
 
 Claypoole is available in the Clojars repository. Just use this leiningen
-dependency: `[com.climate/claypoole "0.2.2"]`.
+dependency: `[com.climate/claypoole "0.3"]`.
 
 ## Why do you use claypoole?
 
@@ -156,6 +156,11 @@ below.)
 
 ## How do I dispose of my threadpools?
 
+*As of Claypoole version 0.3, by default all threadpools are daemon
+threadpools, so they should shut down when your main thread exits. But it's
+still good practice to clean up these resources. Neither the OS nor the JVM
+will take care of them for you!*
+
 The JVM does not automatically garbage collect threads for you. Instead, when
 you're done with your threadpool, you should use `shutdown` to gently shut down
 the pool, or `shutdown!` to kill the threads forcibly.
@@ -172,10 +177,11 @@ Of course, we have provided a convenience macro `with-shutdown!` that will
 ```
 
 Alternately, daemon threads will be automatically killed when the JVM process
-exits. You can create a daemon threadpool via:
+exits. *By default all threadpools are daemon threadpools and will exit when
+the main thread exits!* You can create a non-daemon threadpool via:
 
 ```clojure
-(def pool (cp/threadpool 10 :daemon true))
+(def pool (cp/threadpool 10 :daemon false))
 ```
 
 ## How do I set threadpool options?
@@ -189,7 +195,7 @@ same as the task priority, described below.)
 
 ```clojure
 (def pool (cp/threadpool (cp/ncpus)
-                         :daemon true
+                         :daemon false
                          :thread-priority 3
                          :name "my-pool"))
 ```
@@ -251,6 +257,43 @@ place of a threadpool, and you can use a threadpool just as you would an
 `ExecutorService`. This means you can create custom threadpools and use them
 easily.
 
+## OMG My program isn't exiting!
+
+There are a few cases where threadpools will stop your program from exiting
+that can surprise users. We have endeavored to minimize them, but they can
+still be problems.
+
+### My program doesn't exit until 60 seconds after main exits.
+
+Claypoole actually uses some `clojure.core/future`s.  Unfortunately, those
+threads are from the agent threadpool, and they are not daemon threads. Those
+threads will automatically die 60 seconds after they're used. So if your main
+thread doesn't call either `shutdown-agents` or `System/exit`, those threads
+will keep going for a while!
+
+You'll [experience the same
+thing](http://tech.puredanger.com/2010/06/08/clojure-agent-thread-pools/) if
+you start a `future` or `pmap`--those extra threads have to be explicitly shut
+down, or they'll naturally die after 60 seconds.
+
+The answer is basically to call `(shutdown-agents)` before your main thread
+exits.
+
+_In a future version of Claypoole, we may switch to using a separate daemon
+threadpool for those futures. However, that could have other complications, so
+we are deferring that decision._
+
+### My program doesn't exit ever!
+
+Claypoole now (as of version 0.3) defaults to daemon threadpools, but before
+that threadpools were not daemons. If you have any non-daemon threadpools
+running and your main process exits, those threadpools will never die, and your
+program will hang indefinitely!
+
+The answer is: either use daemon threadpools (which is now the default) or be
+very careful to shut down your threadpools when you're done with them. See the
+above section on disposing of threadpools.
+
 ## Why the name "Claypoole"?
 
 The claypoole library is named after [John Claypoole (Betsy Ross's third

project.clj

@@ -12,7 +12,7 @@
 ;; and limitations under the License.
 
 (defproject com.climate/claypoole
-  "0.2.2"
+  "0.3"
   :description "Claypoole: Threadpool tools for Clojure."
   :url "http://github.com/TheClimateCorporation/claypoole/"
   :license {:name "Apache License Version 2.0"

src/clj/com/climate/claypoole.clj

@@ -67,7 +67,7 @@
   This takes optional keyword arguments:
     :daemon, a boolean indicating whether the threads are daemon threads,
              which will automatically die when the JVM exits, defaults to
-             false)
+             true)
     :name, a string giving the pool name, which will be the prefix of each
              thread name, resulting in threads named \"name-0\",
              \"name-1\", etc. Defaults to \"claypoole-[pool-number]\".
@@ -82,7 +82,8 @@
   ^java.util.concurrent.ScheduledExecutorService
   ;; NOTE: Although I'm repeating myself, I list all the threadpool-factory
   ;; arguments explicitly for API clarity.
-  [n & {:keys [daemon thread-priority] pool-name :name}]
+  [n & {:keys [daemon thread-priority] pool-name :name
+        :or {daemon true}}]
   (impl/threadpool n
                    :daemon daemon
                    :name pool-name

src/clj/com/climate/claypoole/impl.clj

@@ -107,7 +107,8 @@
   "Create a ThreadFactory with keyword options including thread daemon status
   :daemon, the thread name format :name (a string for format with one integer),
   and a thread priority :thread-priority."
-  ^ThreadFactory [& {:keys [daemon thread-priority] pool-name :name}]
+  ^ThreadFactory [& {:keys [daemon thread-priority] pool-name :name
+                     :or {daemon true}}]
   (let [daemon* (boolean daemon)
         pool-name* (or pool-name (default-threadpool-name))
         thread-priority* (or thread-priority

test/com/climate/claypoole_test.clj

@@ -592,6 +592,21 @@
              (check-shuts-off pmap-like))))
 
 
+(deftest test-daemon
+  (let [daemon? (fn [& args] (.isDaemon (Thread/currentThread)))]
+    (testing "threadpools are daemon by default"
+      (cp/with-shutdown! [pool 3]
+        (is (every? boolean
+                    (cp/pmap pool daemon? (range 100))))))
+    (testing "threadpools are daemon by default"
+      (cp/with-shutdown! [pool (cp/threadpool 3)]
+        (is (every? boolean
+                    (cp/pmap pool daemon? (range 100))))))
+    (testing "we can make non-daemon threadpools"
+      (cp/with-shutdown! [pool (cp/threadpool 3 :daemon false)]
+        (is (not-any? boolean
+                      (cp/pmap pool daemon?  (range 100))))))))
+
 (deftest test-future
   (testing "basic future test"
     (cp/with-shutdown! [pool 3]