Merge pull request #640 from chapj2001/dependency-update

Dependency update to ease transition from Astyanax to DataStax Java Driver for Apache Cassandra.

Readme.markdown

@@ -3,7 +3,24 @@ Astyanax
 Astyanax is a high level Java client for [Apache Cassandra](http://cassandra.apache.org).
 Apache Cassandra is a highly available column oriented database.
 
-Astyanax is currently in use at [Netflix](http://movies.netflix.com), but is being [retired](https://medium.com/netflix-techblog/astyanax-retiring-an-old-friend-6cca1de9ac4). Fixes will be made for serious issues, but it is not under active development.
+Astyanax is currently in use at [Netflix](http://movies.netflix.com), but is being [retired](https://medium.com/netflix-techblog/astyanax-retiring-an-old-friend-6cca1de9ac4). 
+Astyanax is no longer under active development, but may receive dependency updates to ease migration away from Astyanax.
+
+In place of Astyanax consider using [DataStax Java Driver for Apache Cassandra](https://github.com/datastax/java-driver) 
+which is under active development and encapsulates a lot of lessons learned from Astyanax. The DataStax driver supports 
+only CQL protocol because Apache Cassandra 4.x drops the Thrift protocol. Switching to the DataStax driver requires 
+using CQL queries, but you'll still be able to access legacy column families via the CQL "with compact storage" option. 
+
+This version of Astyanax shades its dependency on cassandra-all so you can optionally select any version 
+of cassandra-unit you like for unit/integration testing with different versions of Cassandra. When upgrading to this 
+verion of Astyanax you may need to:
+* Explicitly add any cassandra-all transitive dependencies you previously silently depended on via Astayanax transitives.
+* If you were using internal features of Astyanax that (unintentionally) publicly exposed cassandra-all classes, 
+you must either:
+    * Option A (best): Migrate away from Astyanax to [DataStax Java Driver for Apache Cassandra](https://github.com/datastax/java-driver).
+    * Option B (second-best): Use only astyanax-core public interfaces (none of them expose cassandra-all classes).
+    * Option C: Switch your objects from "org.apache.cassandra" to the shaded "com.netflix.astyanax.shaded.org.apache.cassandra"
+     package Astyanax now depends on.
 
 Artifacts
 -------------------------------
@@ -12,19 +29,26 @@ Astyanax jars are published to Maven Central.  As of astyanax 1.56.27 the projec
 
 Required artifacts
 
-|GroupID/Org|ArtifactID/Name|
-| --------- | ------------- |
-|com.netflix.astyanax|astyanax-core|
-|com.netflix.astyanax|astyanax-thrift|
-|com.netflix.astyanax|astyanax-cassandra|
+|GroupID/Org|ArtifactID/Name|Desc|
+| --------- | ------------- |----|
+|com.netflix.astyanax|astyanax-thrift or astyanax-cql|Choose Thrift or CQL protocol. Note Cassandra 4.x+ drops support for Thrift protocol.|
+
+Transitive artifacts (dependencies automatically added via a required artifact)
+
+|GroupID/Org|ArtifactID/Name|Desc|
+| --------- | ------------- |----|
+|com.netflix.astyanax|astyanax-core|Astyanax's public interface.|
+|com.netflix.astyanax|astyanax-cassandra|Cassandra-specific features shared by astyanax-thrift and astyanax-cql|
+|com.netflix.astyanax|astyanax-cassandra-all-shaded|Shaded version of cassandra-all for the few classes used by astyanax-cassandra so projects are free to select arbitrary versions of cassandra-unit for unit/integration testing against newer versions of Cassandra. Hides Astyanax's dependency on cassandra-all by refactoring "org.apache.cassandra" classes to "com.netflix.astyanax.shaded.org.apache.cassandra".|
 
 Optional artifacts
 
-|GroupID/Org|ArtifactID/Name|
-| --------- | ------------- |
-|com.netflix.astyanax|astyanax-queue|
-|com.netflix.astyanax|astyanax-entity-mapper|
-|com.netflix.astyanax|astyanax-recipes|
+|GroupID/Org|ArtifactID/Name|Desc|
+| --------- | ------------- |----|
+|com.netflix.astyanax|astyanax-contrib|Optional integration with other commonly-used Netflix OSS modules.|
+|com.netflix.astyanax|astyanax-queue|Queue implementation backed by Cassandra storage. Use at your own risk -- Cassandra can be a *very bad* storage engine for queues. If you insist on using Cassandra for queues, set a very short TTL and use TimeWindowCompactionStrategy (TWCS).|
+|com.netflix.astyanax|astyanax-entity-mapper||
+|com.netflix.astyanax|astyanax-recipes|Optional implementations of some common patterns. Use at your own risk; some of these are popular but not well-suite for Cassandra for many use cases (looking at you, AllRowsReader).|
 
 Features
 --------
@@ -45,6 +69,13 @@ Documentation
 -------------
 Detailed documentation of Astyanax's features and usage can be found on the [wiki](https://github.com/Netflix/astyanax/wiki) and the [getting started guide](https://github.com/Netflix/astyanax/wiki/Getting-Started).
 
+IntelliJ: currently (June 2018) IntelliJ has a bug which renders an "unfound" error for packages relocated via another 
+module in the same project (e.g. shaded classes from astyanax-cassandra-all-shaded). 
+This affects *only* the Astyanax project itself within IntelliJ; Astyanax still builds perfectly fine via Gradle. 
+Astyanax users are unaffected. For more details see:
+
+* [https://github.com/johnrengelman/shadow/issues/264](https://github.com/johnrengelman/shadow/issues/264) 
+* [https://youtrack.jetbrains.com/issue/IDEA-163411](https://youtrack.jetbrains.com/issue/IDEA-163411)
 
 Ancient History
 ---------------

astyanax-cassandra-all-shaded/build.gradle

@@ -0,0 +1,41 @@
+/**
+ * Shaded version of cassandra-all for the few classes used by astyanax-cassandra. Shading frees projects to select
+ * arbitrary versions of cassandra-unit for unit/integration testing against newer versions of Cassandra.
+ * Hides Astyanax's dependency on cassandra-all by refactoring "org.apache.cassandra" classes to
+ * "com.netflix.astyanax.shaded.org.apache.cassandra".
+ */
+
+plugins {
+    id 'com.github.johnrengelman.plugin-shadow' version '2.0.3'
+    id 'java'
+}
+
+print "Shading cassandra-all for cassandraVersion=${cassandraVersion}\n"
+
+dependencies {
+    compile ("org.apache.cassandra:cassandra-all:$cassandraVersion") {
+        // Exclude all those heavy transitive dependencies because Astyanax doesn't need them
+        transitive = false
+    }
+}
+
+shadowJar {
+    // Don't append default "-all" to end of shaded jar name.
+    classifier = ''
+
+    // Add only the shaded cassandra-all classes
+    dependencies {
+        include(
+                dependency('org.apache.cassandra:cassandra-all::.*')
+        )
+    }
+
+    // Shaded/rename cassandra-all class packages
+    relocate("org.apache.cassandra", "com.netflix.astyanax.shaded.org.apache.cassandra")
+}
+
+tasks.build.dependsOn(shadowJar)
+
+// Create only the shadow jar containing the shaded classes
+tasks.jar.setEnabled(false)
+tasks.jar.dependsOn(tasks.shadowJar)

astyanax-cassandra/build.gradle

@@ -1,19 +1,19 @@
 apply plugin: 'osgi'
 
+/**
+ * Cassandra-specific features shared by astyanax-thrift and astyanax-cql.
+ */
 dependencies {
-    compile  project(':astyanax-core')
-    compile ("org.apache.cassandra:cassandra-all:$cassandraVersion") {
-       force = true
-       transitive = false
-    }
+    compile project(':astyanax-core')
+
+    // Shaded version of cassandra-all for the few simple classes Astyanax actually uses.
+    // Also excludes cassandra-all's transitive dependencies.
+    compile project(path: ':astyanax-cassandra-all-shaded', configuration: 'shadow')
+
     compile "org.xerial.snappy:snappy-java:$snappyVersion"
     compile "org.codehaus.jackson:jackson-mapper-asl:$jacksonVersion"
-    compile "org.apache.cassandra:cassandra-thrift:$cassandraThriftVersion"
     compile "org.apache.servicemix.bundles:org.apache.servicemix.bundles.commons-csv:$commonsCsvVersion"
     compile "org.codehaus.jettison:jettison:$jettisonVersion"
     compile "commons-codec:commons-codec:$commonsCodecVersion"
     compile "org.slf4j:slf4j-api:$slf4jVersion"
-    testCompile "junit:junit:$junitVersion"
 }
-
-

astyanax-cassandra/src/main/java/com/netflix/astyanax/impl/AstyanaxConfigurationImpl.java

@@ -53,9 +53,11 @@ public class AstyanaxConfigurationImpl implements AstyanaxConfiguration {
     private int                 maxThriftSize               = 16384000;
 
     public AstyanaxConfigurationImpl() {
-        partitioners.put(org.apache.cassandra.dht.RandomPartitioner.class.getCanonicalName(), BigInteger127Partitioner.get());
+        partitioners.put("org.apache.cassandra.dht.RandomPartitioner",
+                BigInteger127Partitioner.get());
         try {
-        	partitioners.put(org.apache.cassandra.dht.Murmur3Partitioner.class.getCanonicalName(), Murmur3Partitioner.get());
+        	partitioners.put("org.apache.cassandra.dht.Murmur3Partitioner",
+                    Murmur3Partitioner.get());
         }
         catch (NoClassDefFoundError exception) {
         	// We ignore this for backwards compatiblity with pre 1.2 cassandra.

astyanax-cassandra/src/main/java/com/netflix/astyanax/model/AbstractComposite.java

@@ -27,8 +27,7 @@
 import java.util.Map;
 import java.util.logging.Logger;
 
-import org.apache.cassandra.utils.ByteBufferUtil;
-
+import com.netflix.astyanax.shaded.org.apache.cassandra.utils.ByteBufferUtil;
 import com.google.common.collect.BiMap;
 import com.google.common.collect.ImmutableBiMap;
 import com.google.common.collect.ImmutableClassToInstanceMap;

astyanax-cassandra/src/main/java/com/netflix/astyanax/partitioner/BigInteger127Partitioner.java

@@ -21,7 +21,7 @@
 import java.util.Iterator;
 import java.util.List;
 
-import org.apache.cassandra.dht.RandomPartitioner;
+import com.netflix.astyanax.shaded.org.apache.cassandra.dht.RandomPartitioner;
 
 import com.google.common.collect.Lists;
 import com.netflix.astyanax.Serializer;

astyanax-cassandra/src/main/java/com/netflix/astyanax/partitioner/Murmur3Partitioner.java

@@ -32,7 +32,8 @@ public class Murmur3Partitioner implements Partitioner {
 
     public static final BigInteger ONE     = new BigInteger("1");
 
-    private static final org.apache.cassandra.dht.Murmur3Partitioner partitioner = new org.apache.cassandra.dht.Murmur3Partitioner();
+    private static final com.netflix.astyanax.shaded.org.apache.cassandra.dht.Murmur3Partitioner partitioner =
+            new com.netflix.astyanax.shaded.org.apache.cassandra.dht.Murmur3Partitioner();
     private static final Murmur3Partitioner instance = new Murmur3Partitioner();
 
     public static Partitioner get() {

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/AsciiSerializer.java

@@ -18,7 +18,7 @@
 import java.nio.ByteBuffer;
 import java.nio.charset.Charset;
 
-import org.apache.cassandra.db.marshal.AsciiType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.AsciiType;
 
 /**
  * Almost identical to StringSerializer except we use the US-ASCII character set

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/BigDecimalSerializer.java

@@ -18,7 +18,7 @@
 import java.math.BigDecimal;
 import java.nio.ByteBuffer;
 
-import org.apache.cassandra.db.marshal.DecimalType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.DecimalType;
 
 public class BigDecimalSerializer extends AbstractSerializer<BigDecimal> {
 

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/BigIntegerSerializer.java

@@ -18,7 +18,7 @@
 import java.math.BigInteger;
 import java.nio.ByteBuffer;
 
-import org.apache.cassandra.db.marshal.IntegerType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.IntegerType;
 
 /**
  * Serializer implementation for BigInteger

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/BooleanSerializer.java

@@ -17,7 +17,7 @@
 
 import java.nio.ByteBuffer;
 
-import org.apache.cassandra.db.marshal.BooleanType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.BooleanType;
 
 /**
  * Converts bytes to Boolean and vice versa

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/ByteBufferSerializer.java

@@ -19,7 +19,7 @@
 import java.util.List;
 import java.util.Map;
 
-import org.apache.cassandra.db.marshal.BytesType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.BytesType;
 
 /**
  * The BytesExtractor is a simple identity function. It supports the Extractor

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/ByteSerializer.java

@@ -17,7 +17,7 @@
 
 import java.nio.ByteBuffer;
 
-import org.apache.cassandra.db.marshal.IntegerType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.IntegerType;
 
 public class ByteSerializer extends AbstractSerializer<Byte> {
 

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/BytesArraySerializer.java

@@ -17,7 +17,7 @@
 
 import java.nio.ByteBuffer;
 
-import org.apache.cassandra.db.marshal.BytesType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.BytesType;
 
 import com.netflix.astyanax.Serializer;
 

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/ComparatorType.java

@@ -15,45 +15,43 @@
  */
 package com.netflix.astyanax.serializers;
 
-import com.netflix.astyanax.*;
+import com.netflix.astyanax.Serializer;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.ShadedTypeParser;
+
+import static com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.ShadedTypeParser.SHADED_PREFIX;
 
 /**
  * @author: peter
  */
 public enum ComparatorType {
 
-    ASCIITYPE("org.apache.cassandra.db.marshal.AsciiType", AsciiSerializer.get()), 
-    BYTESTYPE("org.apache.cassandra.db.marshal.BytesType", ByteBufferSerializer.get()), 
-    INTEGERTYPE("org.apache.cassandra.db.marshal.IntegerType", BigIntegerSerializer.get()), 
-    INT32TYPE("org.apache.cassandra.db.marshal.Int32Type", Int32Serializer.get()), 
-    DECIMALTYPE("org.apache.cassandra.db.marshal.DecimalType", BigDecimalSerializer.get()),
-    LEXICALUUIDTYPE("org.apache.cassandra.db.marshal.LexicalUUIDType", UUIDSerializer.get()), 
-    LOCALBYPARTITIONERTYPE("org.apache.cassandra.db.marshal.LocalByPartionerType", ByteBufferSerializer.get()), // FIXME
-    LONGTYPE("org.apache.cassandra.db.marshal.LongType", LongSerializer.get()), 
-    TIMEUUIDTYPE("org.apache.cassandra.db.marshal.TimeUUIDType", TimeUUIDSerializer.get()), 
-    UTF8TYPE("org.apache.cassandra.db.marshal.UTF8Type", StringSerializer.get()), 
-    COMPOSITETYPE("org.apache.cassandra.db.marshal.CompositeType", CompositeSerializer.get()), 
-    DYNAMICCOMPOSITETYPE("org.apache.cassandra.db.marshal.DynamicCompositeType", DynamicCompositeSerializer.get()), 
-    UUIDTYPE("org.apache.cassandra.db.marshal.UUIDType", UUIDSerializer.get()), 
-    COUNTERTYPE("org.apache.cassandra.db.marshal.CounterColumnType", LongSerializer.get()), 
-    DOUBLETYPE("org.apache.cassandra.db.marshal.DoubleType", DoubleSerializer.get()), 
-    FLOATTYPE("org.apache.cassandra.db.marshal.FloatType", FloatSerializer.get()), 
-    BOOLEANTYPE("org.apache.cassandra.db.marshal.BooleanType", BooleanSerializer.get()),
-    DATETYPE("org.apache.cassandra.db.marshal.DateType", DateSerializer.get()),
-    REVERSEDTYPE("org.apache.cassandra.db.marshal.ReversedType", ReversedSerializer.get());
+    ASCIITYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.AsciiType", AsciiSerializer.get()),
+    BYTESTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.BytesType", ByteBufferSerializer.get()),
+    INTEGERTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.IntegerType", BigIntegerSerializer.get()),
+    INT32TYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.Int32Type", Int32Serializer.get()),
+    DECIMALTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.DecimalType", BigDecimalSerializer.get()),
+    LEXICALUUIDTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.LexicalUUIDType", UUIDSerializer.get()),
+    LOCALBYPARTITIONERTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.LocalByPartionerType", ByteBufferSerializer.get()), // FIXME
+    LONGTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.LongType", LongSerializer.get()),
+    TIMEUUIDTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.TimeUUIDType", TimeUUIDSerializer.get()),
+    UTF8TYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.UTF8Type", StringSerializer.get()),
+    COMPOSITETYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.CompositeType", CompositeSerializer.get()),
+    DYNAMICCOMPOSITETYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.DynamicCompositeType", DynamicCompositeSerializer.get()),
+    UUIDTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.UUIDType", UUIDSerializer.get()),
+    COUNTERTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.CounterColumnType", LongSerializer.get()),
+    DOUBLETYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.DoubleType", DoubleSerializer.get()),
+    FLOATTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.FloatType", FloatSerializer.get()),
+    BOOLEANTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.BooleanType", BooleanSerializer.get()),
+    DATETYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.DateType", DateSerializer.get()),
+    REVERSEDTYPE(SHADED_PREFIX + "org.apache.cassandra.db.marshal.ReversedType", ReversedSerializer.get());
 
     private final String className;
     private final String typeName;
     private final Serializer<?> serializer;
 
     private ComparatorType(String className, Serializer<?> serializer) {
         this.className = className;
-        if (className.startsWith("org.apache.cassandra.db.marshal.")) {
-            typeName = className.substring("org.apache.cassandra.db.marshal.".length());
-        }
-        else {
-            typeName = className;
-        }
+        this.typeName = getShadedTypeName(className);
         this.serializer = serializer;
     }
 
@@ -73,15 +71,19 @@ public static ComparatorType getByClassName(String className) {
         if (className == null) {
             return null;
         }
-
         for (ComparatorType type : ComparatorType.values()) {
-            if (type.getClassName().equals(className)) {
-                return type;
-            }
-            if (type.getClassName().equals("org.apache.cassandra.db.marshal." + className)) {
+            if (type.getClassName().equals(getShadedClassName(className))) {
                 return type;
             }
         }
         return null;
     }
+
+    public static String getShadedClassName(String className){
+        return ShadedTypeParser.getShadedClassName(className);
+    }
+
+    public static String getShadedTypeName(String typeName){
+        return ShadedTypeParser.getShadedTypeName(typeName);
+    }
 }

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/DateSerializer.java

@@ -18,7 +18,7 @@
 import java.nio.ByteBuffer;
 import java.util.Date;
 
-import org.apache.cassandra.db.marshal.DateType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.DateType;
 
 /**
  * Converts bytes to Date and vice versa, by first converting the Date to or

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/DoubleSerializer.java

@@ -17,7 +17,7 @@
 
 import java.nio.ByteBuffer;
 
-import org.apache.cassandra.db.marshal.DoubleType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.DoubleType;
 
 /**
  * Uses LongSerializer via translating Doubles to and from raw long bytes form.

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/FloatSerializer.java

@@ -17,7 +17,7 @@
 
 import java.nio.ByteBuffer;
 
-import org.apache.cassandra.db.marshal.FloatType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.FloatType;
 
 /**
  * Uses IntSerializer via translating Float objects to and from raw long bytes

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/GzipStringSerializer.java

@@ -23,7 +23,7 @@
 import java.util.zip.GZIPInputStream;
 import java.util.zip.GZIPOutputStream;
 
-import org.apache.cassandra.db.marshal.UTF8Type;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.UTF8Type;
 import org.apache.commons.codec.binary.StringUtils;
 
 public class GzipStringSerializer extends AbstractSerializer<String> {

astyanax-cassandra/src/main/java/com/netflix/astyanax/serializers/ListSerializer.java

@@ -18,10 +18,8 @@
 import java.nio.ByteBuffer;
 import java.util.List;
 
-import org.apache.cassandra.db.marshal.AbstractType;
-import org.apache.cassandra.db.marshal.ListType;
-
-import com.netflix.astyanax.serializers.AbstractSerializer;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.AbstractType;
+import com.netflix.astyanax.shaded.org.apache.cassandra.db.marshal.ListType;
 
 /**
  * Serializer implementation for generic lists.