1. 一个 大坑：若实例化 JedisShardInfo 时不设置节点名称（name属性），那么当Redis节点列表的顺序发生变化时，会发生“ 键 rehash 现象”

 

使用BTrace追踪redis.clients.util.Sharded的实时状态，验证“Jedis分片机制的一致性哈希算法”实现；

发现一个致命坑：若JedisShardInfo不设置节点名称（name属性），那么当Redis节点列表的顺序发生变化时，会发生“键 rehash 现象”。见Sharded的initialize(...)方法实现：

（I） this.algo.hash("SHARD-" + i + "-NODE-" + n)

【缺点】  大坑：将节点的顺序索引i作为hash的一部分！ 当节点顺序被无意识地调整了，会触发”键 rehash 现象”，那就杯具啦！（"因节点顺序调整而引发rehash"的问题）

（II） this.algo.hash(shardInfo.getName() + "*" + shardInfo.getWeight() + n)

【优点】 这样设计避免了上面"因节点顺序调整而引发rehash"的问题。

【缺点】 坑："节点名称+权重"必须是唯一的，否则节点会出现重叠覆盖！ 同时，"节点名称+ 权重"必须不能被中途改变！

（III） 节点IP:端口号+编号

Memcached Java Client，就是采用这种策略。

【缺点】 因机房迁移等原因，可能导致节点IP发生改变！

（IIII）  唯一节点名称+编号

较好地一致性hash策略是：唯一节点名称+编号，不要考虑权重因素！

long hash = algo.hash(shardInfo.getName() + "*" + n)

 

所以， 在配置Redis服务列表时，必须要设置节点逻辑名称（name属性）。

 

redis.server.list=192.168.6.35:6379: Shard-01,192.168.6.36:6379: Shard-02,192.168.6.37:6379: Shard-03,192.168.6.38:6379: Shard-04

 

相关代码如下所示：

 

 

public class Sharded
    
     > {  public static final int DEFAULT_WEIGHT = 1;  private TreeMap
     
       nodes;  private final Hashing algo;  private final Map
      
       
        , R> resources = new LinkedHashMap
        
         
          , R>(); public Sharded(List
           shards) { this(shards, Hashing.MURMUR_HASH); // MD5 is really not good as we works with 64-bits not 128 } public Sharded(List shards, Hashing algo) { this.algo = algo; initialize(shards); } private void initialize(List shards) { nodes = new TreeMap
             
              (); for (int i = 0; i != shards.size(); ++i) { final S shardInfo = shards.get(i); if (shardInfo.getName() == null) for (int n = 0; n < 160 * shardInfo.getWeight(); n++) { nodes.put(this.algo.hash("SHARD-" + i + "-NODE-" + n), shardInfo); } else for (int n = 0; n < 160 * shardInfo.getWeight(); n++) { nodes.put(this.algo.hash(shardInfo.getName() + "*" + shardInfo.getWeight() + n), shardInfo); } resources.put(shardInfo, shardInfo.createResource()); } } ...}
             
         
        
       
      
     
    

 

2. CustomShardedJedisFactory.destroyObject(PooledObject<ShardedJedis> pooledShardedJedis) 存在“ 客户端连接泄露”问题

 

异常信息如下所示：

 

[2015-01-28 15:33:51] ERROR c.f.f.b.s.r.i.RedisServiceImpl -ShardedJedis close fail   redis.clients.jedis.exceptions.JedisException: Could not return the resource to the pool at redis.clients.util.Pool.returnBrokenResourceObject(Pool.java:85) ~[jedis-2.6.2.jar:na]       at cn.fraudmetrix.forseti.biz.service.redis.impl.CustomShardedJedisPool.returnBrokenResource(CustomShardedJedisPool.java:120) ~[forseti-biz-service-1.0-SNAPSHOT.jar:na]       at cn.fraudmetrix.forseti.biz.service.redis.impl.CustomShardedJedisPool.returnBrokenResource(CustomShardedJedisPool.java:26) ~[forseti-biz-service-1.0-SNAPSHOT.jar:na]       at redis.clients.jedis.ShardedJedis.close(ShardedJedis.java:638) ~[jedis-2.6.2.jar:na]       at cn.fraudmetrix.forseti.biz.service.redis.impl.RedisServiceImpl.close(RedisServiceImpl.java:90) [forseti-biz-service-1.0-SNAPSHOT.jar:na]       at cn.fraudmetrix.forseti.biz.service.redis.impl.RedisServiceImpl.zadd(RedisServiceImpl.java:380) [forseti-biz-service-1.0-SNAPSHOT.jar:na]       at cn.fraudmetrix.forseti.biz.service.redis.impl.RedisServiceImpl.zadd(RedisServiceImpl.java:346) [forseti-biz-service-1.0-SNAPSHOT.jar:na]       ...    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_51]       at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_51]       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]       Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to [B at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:181) ~[jedis-2.6.2.jar:na]       at redis.clients.jedis.BinaryJedis.quit(BinaryJedis.java:136) ~[jedis-2.6.2.jar:na]       at redis.clients.jedis.BinaryShardedJedis.disconnect(BinaryShardedJedis.java:35) ~[jedis-2.6.2.jar:na]at cn.fraudmetrix.forseti.biz.service.redis.impl.CustomShardedJedisFactory.destroyObject(CustomShardedJedisFactory.java:106) ~[forseti-biz-service-1.0-SNAPSHOT.jar:na]       at org.apache.commons.pool2.impl.GenericObjectPool.destroy(GenericObjectPool.java:848) ~[commons-pool2-2.0.jar:2.0]       at org.apache.commons.pool2.impl.GenericObjectPool.invalidateObject(GenericObjectPool.java:626) ~[commons-pool2-2.0.jar:2.0]       at redis.clients.util.Pool.returnBrokenResourceObject(Pool.java:83) ~[jedis-2.6.2.jar:na]       ... 37 common frames omitted

 

 

从异常信息来看，是由于 应用程序无法捕获 运行时 的类型转换异常（“java.lang. ClassCastException: java.lang.Long cannot be cast to [B”）导致关闭操作异常中断，问题的根源代码位于“CustomShardedJedisFactory.destroyObject(CustomShardedJedisFactory.java:106)”。

 

原实现代码 只捕获了 JedisConnectionException 异常，如下所示：

 

 

public void destroyObject(PooledObject
    
      pooledShardedJedis) throws Exception {        final ShardedJedis shardedJedis = pooledShardedJedis.getObject();        shardedJedis.disconnect(); // "链接资源"无法被释放，存在泄露    }
    

 

 

public void disconnect() {    for (Jedis jedis : getAllShards()) {      try {        jedis.quit();      } catch (JedisConnectionException e) {        // ignore the exception node, so that all other normal nodes can release all connections.      }      try {        jedis.disconnect();      } catch (JedisConnectionException e) {        // ignore the exception node, so that all other normal nodes can release all connections.      }    }  }

 

 

修复后 代码捕获了所有的 Exception，就 不存在释放链接时由于异常未捕获而导致链接释放中断。如下所示：

 

public void destroyObject(PooledObject
    
      pooledShardedJedis) throws Exception {        final ShardedJedis shardedJedis = pooledShardedJedis.getObject();        // shardedJedis.disconnect(); // "链接资源"无法被释放，存在泄露        for (Jedis jedis : shardedJedis.getAllShards()) {            try {                // 1. 请求服务端关闭连接                jedis.quit();            } catch (Exception e) {                // ignore the exception node, so that all other normal nodes can release all connections.                // java.lang.ClassCastException: java.lang.Long cannot be cast to [B                // (zadd/zcard 返回 long 类型，而 quit 返回 string 类型。从这里看，上一次的请求结果并未读取)                logger.warn("quit jedis connection for server fail: " + toServerString(jedis), e);            }            try {                // 2. 客户端主动关闭连接                jedis.disconnect();            } catch (Exception e) {                // ignore the exception node, so that all other normal nodes can release all connections.                logger.warn("disconnect jedis connection fail: " + toServerString(jedis), e);            }        }    }