Having issue integrating bloom filter support using JedisPooled

[Ranjeet116966]

I am currently working on integrating Bloom filter support into one of our features. For this, I’m using JedisPooled for the Bloom filter-related operations, as I believe Jedis itself doesn’t support these methods. After deploying these changes to production, I'm encountering an error ("Unexpected end of stream" / JedisConnectionException) on every second request, causing those requests to fail. I’m unsure whether this issue is due to the size of the connection pool or because the timeouts are set too low for Jedis.

I am using JedisPooled in the same way it’s mentioned in the README file, where I create an instance of JedisPooled like this and pass the custom config I have:

JedisPooled jedis = new JedisPooled("localhost", 6379);

Then, I issue commands on this jedis instance.

Jedis version: 4.2.2
Redis version: 7.2.4

Here is my current Jedis configuration:

connection_timeout: 200
read_timeout: 3000
max_total: 40
max_idle: 5
max_wait: 100

Can someone please take a look and help me understand if i am missing something, and if somehow i can get rid of these errors TIA...

[sazzad16]

@Ranjeet116966 Provide a short code that reproduces the issue and add the full stacktrace.

[Ranjeet116966]

Here is the full stack trace, will provide the code snippet in sometime, Actually i am integrating bloom filter in some gem, that we consume on some other repo, all the config that we need to initialise the connection pool would be passed from consumer end, so will have to create a short code file to replicate the same behaviour

ERROR jruby.app- Connection handling error: Unexpected end of stream. redis.clients.jedis.util.RedisInputStream.ensureFill(redis/clients/jedis/util/RedisInputStream.java:205) redis.clients.jedis.util.RedisInputStream.readByte(redis/clients/jedis/util/RedisInputStream.java:46) redis.clients.jedis.Protocol.process(redis/clients/jedis/Protocol.java:126) redis.clients.jedis.Protocol.read(redis/clients/jedis/Protocol.java:192) redis.clients.jedis.Connection.readProtocolWithCheckingBroken(redis/clients/jedis/Connection.java:316) redis.clients.jedis.Connection.getOne(redis/clients/jedis/Connection.java:298) redis.clients.jedis.Connection.executeCommand(redis/clients/jedis/Connection.java:123) redis.clients.jedis.executors.DefaultCommandExecutor.executeCommand(redis/clients/jedis/executors/DefaultCommandExecutor.java:24) redis.clients.jedis.UnifiedJedis.executeCommand(redis/clients/jedis/UnifiedJedis.java:148) redis.clients.jedis.UnifiedJedis.bfExists(redis/clients/jedis/UnifiedJedis.java:3923) java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:498) org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:470) org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:328)

[Ranjeet116966]

Here is the basic code snippet you can refer to, this is how i have also implemented, and i am getting the above error after deploying this code to prod

require 'java'

require 'redis.clients:jedis-4.2.2.jar'
require 'org.slf4j:slf4j-api-2.0.13.jar'
require 'org.apache.logging.log4j:log4j-slf4j2-impl-2.23.1.jar'

java_import 'redis.clients.jedis.JedisPooled'
java_import 'redis.clients.jedis.JedisPoolConfig'
java_import 'redis.clients.jedis.bloom.RedisBloomCommands'

connection_timeout = 200
read_timeout = 3000

def initialize_pool(connection_timeout, read_timeout)
  jedis_pool_config = JedisPoolConfig.new
  jedis_pool_config.setMaxTotal(5)
  jedis_pool_config.setMaxIdle(5)
  jedis_pool_config.setMaxWaitMillis(100)
  jedis_pool_config.setBlockWhenExhausted(true)

  JedisPooled.new(
    jedis_pool_config,
    java.net.URI.create("redis://localhost:6380"),
    connection_timeout,
    read_timeout
  )
end

def execute_bloom_filter_command(pool)
  pool.bfAdd('bloom_filter_key', 'item')
  puts pool.bfExists('bloom_filter_key', 'item')
end

pool = initialize_pool(connection_timeout, read_timeout)
execute_bloom_filter_command(pool)

[sazzad16]

@Ranjeet116966 Couldn't reproduce your issue. Can't even guess what could go wrong within localhost.

• What's the timeout in Redis server? I.g. redis-cli ==> CONFIG GET timeout or Jedis ==> configGet("timeout").
• Could you try increasing the connection timeout? I.e. as high the read_timeout. (Seems doesn't matter in case, but just to be safe.)
• Could you try an updated Jedis version? If the latest version is not possible, at least Jedis 4.4.8?

[Ranjeet116966]

Reproducing the error is difficult because it's not very frequent, some of the request are passing and some request are failing so error is very inconsistent in my testing env as well.

• Redis timeout is set to 0 i think

CONFIG GET timeout
1) "timeout"
2) "0"

Will try increasing theconnect_timeoutand if that doesn't help will also upgrade the jedis version, Did we change anything related to this issue, i didn't find anything like that in release notes