首页 > 编程 > Java > 正文

Java中LocalCache本地缓存实现代码

2019-11-26 12:20:12
字体:
来源:转载
供稿:网友

前言

本次分享探讨java平台的本地缓存,是指占用JVM的heap区域来缓冲存储数据的缓存组件。

一、本地缓存应用场景

localcache有着极大的性能优势:

1. 单机情况下适当使用localcache会使应用的性能得到很大的提升。

2. 集群环境下对于敏感性要求不高的数据可以使用localcache,只配置简单的失效机制来保证数据的相对一致性。

哪些数据可以存储到本地缓存?

1.访问频繁的数据;

2.静态基础数据(长时间内不变的数据);

3.相对静态数据(短时间内不变的数据)。

二、java本地缓存标准

Java缓存新标准(javax.cache),这个标准由JSR107所提出,已经被包含在Java EE 7中。

特性:
1.原子操作,跟java.util.ConcurrentMap类似
2.从缓存中读取
3.写入缓存
4.缓存事件监听器
5.数据统计
6.包含所有隔离(ioslation)级别的事务
7.缓存注解(annotations)
8.保存定义key和值类型的泛型缓存
9.引用保存(只适用于堆缓存)和值保存定义

但目前应用不是很普遍。

三、java开源缓存框架

比较有名的本地缓存开源框架有:

1.EHCache

EHCache是一个纯java的在进程中的缓存,它具有以下特性:快速,简单,为Hibernate2.1充当可插入的缓存,最小的依赖性,全面的文档和测试。

BUG: 过期失效的缓存元素无法被GC掉,时间越长缓存越多,内存占用越大,导致内存泄漏的概率越大。

2.OSCache

OSCache有以下特点:缓存任何对象,你可以不受限制的缓存部分jsp页面或HTTP请求,任何java对象都可以缓存。拥有全面的API--OSCache API给你全面的程序来控制所有的OSCache特性。永久缓存--缓存能随意的写入硬盘,因此允许昂贵的创建(expensive-to-create)数据来保持缓存,甚至能让应用重启。支持集群--集群缓存数据能被单个的进行参数配置,不需要修改代码。缓存记录的过期--你可以有最大限度的控制缓存对象的过期,包括可插入式的刷新策略(如果默认性能不需要时)。

3.JCache

Java缓存新标准(javax.cache)

4.cache4j

cache4j是一个有简单API与实现快速的Java对象缓存。它的特性包括:在内存中进行缓存,设计用于多线程环境,两种实现:同步与阻塞,多种缓存清除策略:LFU, LRU, FIFO,可使用强引用。

5.ShiftOne

ShiftOne Java Object Cache是一个执行一系列严格的对象缓存策略的Java lib,就像一个轻量级的配置缓存工作状态的框架。

6.WhirlyCache

Whirlycache是一个快速的、可配置的、存在于内存中的对象的缓存。 

四、LocalCache实现

1、LocalCache简介

LocalCache是一个精简版本地缓存组件,有以下特点:

1.  有容量上限maxCapacity;
2.  缓存达到容量上限时基于LRU策略来移除缓存元素;
3.  缓存对象的生命周期(缓存失效时间)由调用方决定;
4.  缓存对象失效后,将会有定时清理线程来清理掉,不会导致内存泄漏。
5.  性能比Ehcache稍强。

2、总体设计

LocalCache总体设计:

1.  缓存元素 CacheElement;
2.  缓存容器 LRULinkedHashMap;
3.  缓存接口 Cache;
4.  缓存组件实现 LocalCache。

3、详细设计

1.  CacheElement设计

/** * 缓存元素 * */public class CacheElement { private Object key; private Object value; private long createTime; private long lifeTime; private int hitCount; public CacheElement() { } public CacheElement(Object key ,Object value) { this.key = key; this.value = value; this.createTime = System.currentTimeMillis(); }  public Object getKey() { return key; } public void setKey(Object key) { this.key = key; } public Object getValue() { hitCount++; return value; } public void setValue(Object value) { this.value = value; } public long getCreateTime() { return createTime; } public void setCreateTime(long createTime) { this.createTime = createTime; } public int getHitCount() { return hitCount; } public void setHitCount(int hitCount) { this.hitCount = hitCount; } public long getLifeTime() { return lifeTime; } public void setLifeTime(long lifeTime) { this.lifeTime = lifeTime; }  public boolean isExpired() { boolean isExpired = System.currentTimeMillis() - getCreateTime() > getLifeTime(); return isExpired; } /* * (non-Javadoc) * @see java.lang.Object#toString() */ public String toString() { StringBuffer sb = new StringBuffer(); sb.append("[ key=").append(key).append(", isExpired=").append(isExpired())  .append(", lifeTime=").append(lifeTime).append(", createTime=").append(createTime)  .append(", hitCount=").append(hitCount)  .append(", value=").append(value).append(" ]"); return sb.toString(); }  /* * (non-Javadoc) * @see java.lang.Object#hashCode() */ public final int hashCode(){ if(null == key){  return "".hashCode(); } return this.key.hashCode(); }  /* * (non-Javadoc) * @see java.lang.Object#equals(java.lang.Object) */ public final boolean equals(Object object) { if ((object == null) || (!(object instanceof CacheElement))) {  return false; } CacheElement element = (CacheElement) object; if ((this.key == null) || (element.getKey() == null)) {  return false; } return this.key.equals(element.getKey()); }}

2.  LRULinkedHashMap实现

import java.util.LinkedHashMap;import java.util.Set;import java.util.concurrent.locks.Lock;import java.util.concurrent.locks.ReentrantLock;/** * 实现 LRU策略的 LinkedHashMap * * @param <K> * @param <V> */public class LRULinkedHashMap<K, V> extends LinkedHashMap<K, V> {  protected static final long serialVersionUID = 2828675280716975892L;  protected static final int DEFAULT_MAX_ENTRIES = 100;  protected final int initialCapacity;  protected final int maxCapacity;  protected boolean enableRemoveEldestEntry = true;//是否允许自动移除比较旧的元素(添加元素时)  protected static final float DEFAULT_LOAD_FACTOR = 0.8f;  protected final Lock lock = new ReentrantLock();   public LRULinkedHashMap(int initialCapacity)   {    this(initialCapacity, DEFAULT_MAX_ENTRIES);  }    public LRULinkedHashMap(int initialCapacity ,int maxCapacity)   {    //set accessOrder=true, LRU    super(initialCapacity, DEFAULT_LOAD_FACTOR, true);         this.initialCapacity = initialCapacity;     this.maxCapacity = maxCapacity;   }  /*   *  (non-Javadoc)   * @see java.util.LinkedHashMap#removeEldestEntry(java.util.Map.Entry)   */  protected boolean removeEldestEntry(java.util.Map.Entry<K, V> eldest)   {     return enableRemoveEldestEntry && ( size() > maxCapacity );  }  /* *  (non-Javadoc) * @see java.util.LinkedHashMap#get(java.lang.Object) */  public V get(Object key)   {     try {       lock.lock();       return super.get(key);     }     finally {       lock.unlock();     }   }   /*   *  (non-Javadoc)   * @see java.util.HashMap#put(java.lang.Object, java.lang.Object)   */  public V put(K key, V value)   {     try {       lock.lock();       return super.put(key, value);     }     finally {       lock.unlock();     }   }     /*   * (non-Javadoc)   * @see java.util.HashMap#remove(java.lang.Object)   */  public V remove(Object key) {    try {       lock.lock();      return super.remove(key);    }     finally {       lock.unlock();     }  }    /*   * (non-Javadoc)   * @see java.util.LinkedHashMap#clear()   */  public void clear() {   try {        lock.lock();       super.clear();     }      finally {        lock.unlock();     }  }    /*   * (non-Javadoc)   * @see java.util.HashMap#keySet()   */  public Set<K> keySet() {   try {       lock.lock();      return super.keySet();    }     finally {       lock.unlock();    }  }    public boolean isEnableRemoveEldestEntry() { return enableRemoveEldestEntry; } public void setEnableRemoveEldestEntry(boolean enableRemoveEldestEntry) { this.enableRemoveEldestEntry = enableRemoveEldestEntry; } public int getInitialCapacity() { return initialCapacity; } public int getMaxCapacity() { return maxCapacity; }} 

3.  Cache接口设计

/** * 缓存接口 * */public interface Cache {  /** * 获取缓存 * @param key * @return */ public <T> T getCache(Object key);  /** * 缓存对象 * @param key * @param value * @param milliSecond 缓存生命周期(毫秒) */ public void putCache(Object key, Object value ,Long milliSecond);  /** * 缓存容器中是否包含 key  * @param key * @return */ public boolean containsKey(Object key);  /** * 缓存列表大小 * @return */ public int getSize();  /** * 是否启用缓存 */ public boolean isEnabled(); /** * 启用 或 停止 * @param enable */ public void setEnabled(boolean enabled);  /** * 移除所有缓存 */ public void invalidateCaches();  /** * 移除 指定key缓存 * @param key */ public void invalidateCache(Object key);}

4.  LocalCache实现

import java.util.Date;import java.util.Iterator;import java.util.Random;import org.slf4j.Logger;import org.slf4j.LoggerFactory;/** * 本地缓存组件 */public class LocalCache implements Cache{ private Logger logger = LoggerFactory.getLogger(this.getClass());  private LRULinkedHashMap<Object, CacheElement> cacheMap;  protected boolean initFlag = false;//初始化标识  protected final long defaultLifeTime = 5 * 60 * 1000;//5分钟 protected boolean warnLongerLifeTime = false;  protected final int DEFAULT_INITIAL_CAPACITY = 100; protected final int DEFAULT_MAX_CAPACITY = 100000;  protected int initialCapacity = DEFAULT_INITIAL_CAPACITY;//初始化缓存容量 protected int maxCapacity = DEFAULT_MAX_CAPACITY;//最大缓存容量 protected int timeout = 20;//存取缓存操作响应超时时间(毫秒数)  private boolean enabled = true;  private Thread gcThread = null; private String lastGCInfo = null;//最后一次GC清理信息{ size, removeCount, time ,nowTime} private boolean logGCDetail = false;//记录gc清理细节  private boolean enableGC = true;//是否允许清理的缓存(添加元素时) private int gcMode = 0;//清理过期元素模式 { 0=迭代模式 ; 1=随机模式 } private int gcIntervalTime = 2 * 60 * 1000;//间隔时间(分钟)  private boolean iterateScanAll = true;//是否迭代扫描全部 private float gcFactor = 0.5F;//清理百分比 private int maxIterateSize = DEFAULT_MAX_CAPACITY/2;//迭代模式下一次最大迭代数量 private volatile int iterateLastIndex = 0;//最后迭代下标 private int maxRandomTimes = 100;//随机模式下最大随机次数   protected final static Random random = new Random(); private static LocalCache instance = new LocalCache(); public static LocalCache getInstance() { return instance; } private LocalCache(){ this.init(); }  protected synchronized void init() { if(initFlag){  logger.warn("init repeat.");  return ; }  this.initCache();  this.startGCDaemonThread();  initFlag = true;  if(logger.isInfoEnabled()){  logger.info("init -- OK"); } }  private void startGCDaemonThread(){ if(initFlag){  return ; }  this.maxIterateSize = maxCapacity /2; try{  this.gcThread = new Thread() {  public void run() {   logger.info("[" + (Thread.currentThread().getName()) + "]start...");   //sleep   try {   Thread.sleep(getGcIntervalTime() < 30000 ? 30000 : getGcIntervalTime());   } catch (Exception e) {   e.printStackTrace();   }   while( true ){   //gc   gc();   //sleep   try {    Thread.sleep(getGcIntervalTime() < 30000 ? 30000 : getGcIntervalTime());   } catch (Exception e) {    e.printStackTrace();   }   }  }  };  this.gcThread.setName("localCache-gcThread");  this.gcThread.setDaemon(true);  this.gcThread.start();    if(logger.isInfoEnabled()){  logger.info("startGCDaemonThread -- OK");  } }catch(Exception e){  logger.error("[localCache gc]DaemonThread -- error: " + e.getMessage(), e); } }  private void initCache(){ if(initFlag){  return ; }  initialCapacity = (initialCapacity <= 0 ? DEFAULT_INITIAL_CAPACITY : initialCapacity); maxCapacity = (maxCapacity < initialCapacity ? DEFAULT_MAX_CAPACITY : maxCapacity);  cacheMap = new LRULinkedHashMap<Object, CacheElement>(initialCapacity ,maxCapacity);  if(logger.isInfoEnabled()){  logger.info("initCache -- OK"); } }  /* * (non-Javadoc) */ @SuppressWarnings("unchecked") public <T> T getCache(Object key) { if(!isEnabled()){  return null; } long st = System.currentTimeMillis();  T objValue = null; CacheElement cacheObj = cacheMap.get(key);  if (isExpiredCache(cacheObj)) {  cacheMap.remove(key); }else {  objValue = (T) (cacheObj == null ? null : cacheObj.getValue()); }  long et = System.currentTimeMillis(); if((et - st)>timeout){  if(this.logger.isWarnEnabled()){  this.logger.warn("getCache_timeout_" + (et - st) + "_[" + key + "]");  } }  if(logger.isDebugEnabled()){  String message = ("get( " + key + ") return: " + objValue);  logger.debug(message); } return objValue; } /* * (non-Javadoc) */ public void putCache(Object key, Object value ,Long lifeTime) { if(!isEnabled()){  return; } Long st = System.currentTimeMillis();  lifeTime = (null == lifeTime ? defaultLifeTime : lifeTime); CacheElement cacheObj = new CacheElement(); cacheObj.setCreateTime(System.currentTimeMillis()); cacheObj.setLifeTime(lifeTime); cacheObj.setValue(value); cacheObj.setKey(key); cacheMap.put(key, cacheObj);  long et = System.currentTimeMillis(); if((et - st)>timeout){  if(this.logger.isWarnEnabled()){  this.logger.warn("putCache_timeout_" + (et - st) + "_[" + key + "]");  } }  if(logger.isDebugEnabled()){  String message = ("putCache( " + cacheObj + " ) , 耗时 " + (et - st) + "(毫秒).");  logger.debug(message); } if(lifeTime > defaultLifeTime && this.isWarnLongerLifeTime()){  if(logger.isWarnEnabled()){  String message = ("LifeTime[" + (lifeTime/1000) + "秒] too long for putCache(" + cacheObj + ")");  logger.warn(message);  } } }  /** * key 是否过期 * @param key * @return */ protected boolean isExpiredKey(Object key) { CacheElement cacheObj = cacheMap.get(key); return this.isExpiredCache(cacheObj); }  /** * cacheObj 是否过期 * @param key * @return */ protected boolean isExpiredCache(CacheElement cacheObj) { if (cacheObj == null) {  return false; } return cacheObj.isExpired(); }  /* * (non-Javadoc) */ public void invalidateCaches(){ try{  cacheMap.clear(); }catch(Exception e){  e.printStackTrace(); } }  /* * (non-Javadoc) */ public void invalidateCache(Object key){ try{  cacheMap.remove(key); }catch(Exception e){  e.printStackTrace(); } }  /* * (non-Javadoc) */ public boolean containsKey(Object key) { return cacheMap.containsKey(key); } /* * (non-Javadoc) */ public int getSize() { return cacheMap.size(); } /* * (non-Javadoc) */ public Iterator<Object> getKeyIterator() { return cacheMap.keySet().iterator(); } /* * (non-Javadoc) */ public boolean isEnabled() { return this.enabled; } /* * (non-Javadoc) */ public void setEnabled(boolean enabled) { this.enabled = enabled; if(!this.enabled){  //清理缓存  this.invalidateCaches(); } }   /**   * 清理过期缓存   */  protected synchronized boolean gc(){      if(!isEnableGC()){   return false;   }      try{      iterateRemoveExpiredCache();    }catch(Exception e){  logger.error("gc() has error: " + e.getMessage(), e); }   return true;  }   /** * 迭代模式 - 移除过期的 key * @param exceptKey */ private void iterateRemoveExpiredCache(){ long startTime = System.currentTimeMillis();   int size = cacheMap.size(); if(size ==0){  return; }  int keyCount = 0; int removedCount = 0 ;  int startIndex = 0; int endIndex = 0;  try{  Object [] keys = cacheMap.keySet().toArray();  keyCount = keys.length;  int maxIndex = keyCount -1 ;    //初始化扫描下标  if(iterateScanAll){  startIndex = 0;  endIndex = maxIndex;  }else {  int gcThreshold = this.getGcThreshold();  int iterateLen = gcThreshold > this.maxIterateSize ? this.maxIterateSize : gcThreshold;    startIndex = this.iterateLastIndex;  startIndex = ( (startIndex < 0 || startIndex > maxIndex) ? 0 : startIndex );  endIndex = (startIndex + iterateLen);  endIndex = (endIndex > maxIndex ? maxIndex : endIndex);  }    //迭代清理  boolean flag = false;  for(int i=startIndex; i<= endIndex; i++){  flag = this.removeExpiredKey(keys[i]);  if(flag){   removedCount++;  }  }    this.iterateLastIndex = endIndex;  keys = null;   }catch(Exception e){  logger.error("iterateRemoveExpiredCache -- 移除过期的 key时出现异常: " + e.getMessage(), e); }  long endTime = System.currentTimeMillis();  StringBuffer sb = new StringBuffer(); sb.append("iterateRemoveExpiredCache [ size: ").append(size).append(", keyCount: ").append(keyCount)  .append(", startIndex: ").append(startIndex).append(", endIndex: ").append(iterateLastIndex)  .append(", removedCount: ").append(removedCount).append(", currentSize: ").append(this.cacheMap.size())  .append(", timeConsuming: ").append(endTime - startTime).append(", nowTime: ").append(new Date())  .append(" ]"); this.lastGCInfo = sb.toString();  if(logger.isInfoEnabled()){  logger.info("iterateRemoveExpiredCache -- 清理结果 -- "+ lastGCInfo); } }  /** * 随机模式 - 移除过期的 key */ private void randomRemoveExpiredCache(){ long startTime = System.currentTimeMillis();   int size = cacheMap.size(); if(size ==0){  return; }  int removedCount = 0 ; try{  Object [] keys = cacheMap.keySet().toArray();  int keyCount = keys.length;    boolean removeFlag = false;    int removeRandomTimes = this.getGcThreshold();    removeRandomTimes = ( removeRandomTimes > this.getMaxRandomTimes() ? this.getMaxRandomTimes() : removeRandomTimes );  while(removeRandomTimes-- > 0){    int index = random.nextInt(keyCount);  boolean flag = this.removeExpiredKey(keys[index]);  if(flag){   removeFlag = true;   removedCount ++;  }  }  //尝试 移除 首尾元素  if(!removeFlag){   this.removeExpiredKey(keys[0]);   this.removeExpiredKey(keys[keyCount-1]);  }  keys=null;   }catch(Exception e){      logger.error("randomRemoveExpiredCache -- 移除过期的 key时出现异常: " + e.getMessage(), e); } long endTime = System.currentTimeMillis();  StringBuffer sb = new StringBuffer(); sb.append("randomRemoveExpiredCache [ size: ").append(size).append(", removedCount: ").append(removedCount)  .append(", currentSize: ").append(this.cacheMap.size()).append(", timeConsuming: ").append(endTime - startTime)  .append(", nowTime: ").append(new Date())  .append(" ]"); this.lastGCInfo = sb.toString();  if(logger.isInfoEnabled()){  logger.info("randomRemoveExpiredCache -- 清理结果 -- "+ lastGCInfo);  } }  private boolean removeExpiredKey(Object key){ boolean flag = false;  CacheElement cacheObj = null; if(null != key){  try{  cacheObj = cacheMap.get(key);  boolean isExpiredCache = this.isExpiredCache(cacheObj);  if(isExpiredCache){   cacheMap.remove(key);   flag = true;  }  }catch(Exception e){  logger.error("removeExpired(" + key + ") -- error: " + e.getMessage(), e);  } }  if(!flag && logGCDetail){  this.logger.warn("removeExpiredKey(" + key + ") return [" + flag + "]--" + cacheObj); }  return flag; }  public int getInitialCapacity() { return initialCapacity; } public int getMaxCapacity() { return maxCapacity; } public int getGcMode() { return gcMode; } public void setGcMode(int gcMode) { this.gcMode = gcMode; } public int getGcIntervalTime() { return gcIntervalTime; } public void setGcIntervalTime(int gcIntervalTime) { this.gcIntervalTime = gcIntervalTime; } public boolean isEnableGC() { return enableGC; } public void setEnableGC(boolean enableGC) { this.enableGC = enableGC; }  public boolean isIterateScanAll() { return iterateScanAll; } public void setIterateScanAll(boolean iterateScanAll) { this.iterateScanAll = iterateScanAll; } public float getGcFactor() { return gcFactor; } public void setGcFactor(float gcFactor) { this.gcFactor = gcFactor; }  /** * gc 阀值 * @return */ public int getGcThreshold() { int threshold = (int)( this.cacheMap.getMaxCapacity() * gcFactor ); return threshold; } public String getLastGCInfo() { return lastGCInfo; } public void setLastGCInfo(String lastGCInfo) { this.lastGCInfo = lastGCInfo; }  public boolean isLogGCDetail() { return logGCDetail; } public void setLogGCDetail(boolean logGCDetail) { this.logGCDetail = logGCDetail; } public int getTimeout() { return timeout; } public void setTimeout(int timeout) { this.timeout = timeout; } public int getMaxIterateSize() { return maxIterateSize; } public void setMaxIterateSize(int maxIterateSize) { this.maxIterateSize = maxIterateSize; } public int getMaxRandomTimes() { return maxRandomTimes; } public void setMaxRandomTimes(int maxRandomTimes) { this.maxRandomTimes = maxRandomTimes; } public boolean isInitFlag() { return initFlag; } public long getDefaultLifeTime() { return defaultLifeTime; } public boolean isWarnLongerLifeTime() { return warnLongerLifeTime; } public void setWarnLongerLifeTime(boolean warnLongerLifeTime) { this.warnLongerLifeTime = warnLongerLifeTime; } //======================== dynMaxCapacity ======================== private int dynMaxCapacity = maxCapacity; public int getDynMaxCapacity() { return dynMaxCapacity; } public void setDynMaxCapacity(int dynMaxCapacity) { this.dynMaxCapacity = dynMaxCapacity; } public void resetMaxCapacity(){ if(dynMaxCapacity > initialCapacity && dynMaxCapacity != maxCapacity){    if(logger.isInfoEnabled()){  logger.info("resetMaxCapacity( " + dynMaxCapacity + " ) start...");  }    synchronized(cacheMap){  LRULinkedHashMap<Object, CacheElement> cacheMap0 = new LRULinkedHashMap<Object, CacheElement>(initialCapacity ,dynMaxCapacity);  cacheMap.clear();  cacheMap = cacheMap0;  this.maxCapacity = dynMaxCapacity;  }    if(logger.isInfoEnabled()){  logger.info("resetMaxCapacity( " + dynMaxCapacity + " ) OK.");  } }else {  if(logger.isWarnEnabled()){  logger.warn("resetMaxCapacity( " + dynMaxCapacity + " ) NO.");  } } } //======================== showCacheElement ======================== private String showCacheKey; public String getShowCacheKey() { return showCacheKey; } public void setShowCacheKey(String showCacheKey) { this.showCacheKey = showCacheKey; } public Object showCacheElement(){ Object v = null;  if(null != this.showCacheKey){  v = cacheMap.get(showCacheKey); }  return v; }}

以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持武林网。

发表评论 共有条评论
用户名: 密码:
验证码: 匿名发表