2016年5月5日星期四

MSST16 Session 4: Spotlight on Flash memory and Solid-State Drives

Adaptive policies for balancing performance and lifetime of mixed SSD arrays through workload sampling

high-end SSD: cache
Low-end SSD: main storage 

1 high-end SSD cache for 3 low-end SSD: high-end SSD life is 1.47 years versus low-end 6.34 years assuming LRU cache policy

problem: high-end SSD cache can wear out faster than low-end SSDs main storage 

approach: balance the performance and lifetime at the same time 
metric: optimize latency over lifetime (less is better) 

selective caching policies ---> decide cache policy based on request size and hotness 

REAL: A Retention Error Aware LDPC Decoding Scheme to Improve NAND Flash Read Performance 

error correction codes: BCH, LDPC

Analytic models for flash-based SSD performance when subject to trimming 

SSD structures: N blocks, b pages per block, unit of data exchange is a page, page has 3 possible states: eras, valid or invalid

data can only be wrtten on pages in erase state
erase can be performed on whole block only 

assume j valid pages on a victim block with probability pj,
write amplification A equals 
A = b/(b - sum(j*pj)) 

prior work: mostly assumes uniform random writes and Rosenblum(hot/cold) workloads 
               exact (closed form) results when N -> infinity 
                  1. greedy is optimzed under random writes, d-choices close to optimal (for d as small as 10) 
                     2. increaseing hotness worsens WA in case of single WF (as no hot/cold data sepeartion takes place) 
                     3. Double WF (seperates writes triggered by hot and GC): WA decreases with hotness (as partial hot/cold data separation takes place) 
                      However, they all assume no trimming

How do we model trim behavior? 
      
Main takeaway: 
trimming results in effective load (utilization)

Reducing Write Amplification of Flash Storage through Cooperative Data Management with NVM

write amplification and GC causes SSD performance fluctuation 

in traditional systems, all live pages need to be copied to another block whiling erasing
however, CDM skips coping 
"removable" state: can be erased if the data needed to be copied into 


issue 1: consistency ---> file system needs to be modified 
issue 2: communication overhead -> events in cache and storage should be notified to each other synchronously --> use NVM-e to piggyback 

NV-cache as in-storage cache

evaluation: 
CDM reduces write-amplification by 20x, improves response time as well

Exploiting Latency Variation for Access Conflict Reduction of NAND Flash Memory

motivation: 
ECC complexity, ECC capability and read speed tradeoffs: high sensing level means preciser memory and higher ECC capability 
program size and write speed tradeoff: 
process variable and retention variation leed to speed variation 


hotness-aware write scheduling: 
retention aware read scheduling
write: size-based predicted hotness
read: 

evaluation: 







MSST'16 session 3 Store More, Longer, and for Less: Deduplication and Archival Systems

A Long Term User-Centric Analysis of Deduplication Patterns

study a dataset of 21 months, 1 snapshot per user per day
tracer.filesystems.org 

a lot of small files (< 1M), but a few large files consume most of the space
in general, small files achieve higher deduplication ratio than large files 

per-user deduplication ratio, redundancy (across users),  differs a lot  

Lazy Exact Deduplication

postpone disk lookups (fingerprints lookup) until we can do them in a batch

Sorted Deduplication: How to Process Thousands of Backup Streams

requirement is changing: a few large streams ---> many streams (e.g., cloud backup)

Effects of Prolonged Media Usage and Long-term Planning on Archival Systems

preserving data for ~100 to ~1000 years

question: 
when do you retire/replace media?
how long do you plan for?

Failure scenarios: device failures and economic failure 

1. should media be used past their manufacture suggested service life or warranty period? 
    (for archival data disk might last longer) 

have a model to model the purchase, maintaining and retiring phase to calculate cost 






MSST'16: It's never too fast: storage performance enhancements

Pfimbi: Accelerating Big Data Jobs Through Flow-Controlled Data Replication

in HDFS, synchronous replication (in pipeline) has performance bottlenecks, and seldomly helps application performance
   - only 2% of data was read within 5mins of being written
 
So do asynchronous replication
Need to use flow control to manage congestion as well

ManyLogs: Improved CMR/SMR Disk Bandwidth and Faster Durability with Scattered Logs

problem: small durable writes severely impacts bandwidth of other users (e.g., sequential reader) 

in this case, data journaling outperforms ordered journaling!

Ordered journaling: efficient for large writes
data journaling:efficient for small writes (less seeks) 
previous work: adaptive journaling (ATC'05) 

many logs, small writes to the nearest log (to the current head)  

where to put logs on the disk? reserve 10MB for every platter (?)

checkpointing: lazy instead of every 5 seconds for many-logs 








2016年1月31日星期日

Cluster setup in hbase (zz)

Cluster setup in hbase
Before Starting hbase cluster
To configure HBase, we need to have a running Hadoop cluster, which will be the storage for hbase(Hbase store data in HDFS). Please refere to   Installing and configuring hadoop cluster .And plese make sure that user name of all machines and the path where the hbase is installed are same in all machines.In my case user is hduser.
These are the steps ,how to setup and  run Hbase cluster.We have  build hbase cluster using three Ubuntu machine.A distributed HBase depends on a running ZooKeeper cluster.we are using default ZooKeeper cluster, which is manage by Hbase.
There are basically three type of node.
1. Hbase Master:- The HbaseMaster is responsible for assigning regions to HbaseRegionserver, monitors the health of each HbaseRegionserver.
2. Zookeeper: – For any distributed application, ZooKeeper is a centralized service for maintaining configuration information, naming, providing        distributed synchronization, and providing group services.
3Hbase Regionserver:- The HbaseRegionserver is responsible for handling client read and write requests. It communicates with the Hbasemaster to get a list of regions to serve and to tell the master that it is alive.
In our Example, one machine in the cluster is designated as Hbase master and Zookeeper. The rest of machine in the cluster act as a Regionserver.

INSTALLING AND CONFIGURING HBASE MASTER
1. Download hbase-1.1.2tar.gz from http://www.apache.org/dyn/closer.cgi/hbase/ and extract it in some directory in your computer. Now this path is called      as $HBASE_INSTALL_DIR.
2. Edit the file /etc/hosts on the master machine and add the following lines.
192.168.35.16 bsw-HbaseMaster     bsw-HbaseMaster
     Hbase Master and Hadoop Namenode(master machine in hadoop clustering) is configure on same machine
 192.168.35.17 bsw-data1
 192.168.35.25 bsw-data2
Note: Run the command “ping bsw-HbaseMaster”. This command is run to check whether the bsw-HbaseMaster machine IP is being resolved to actual IP not localhost IP.
Here bsw-data1 and bsw-data2 are the machine where region server is running and bsw-HbaseMaster is the machine where hbase-master is running
3. We have needed to configure password less login from bsw-HbaseMaster to all regionserver machines.
          Execute the following commands on bsw-HbaseMaster  machine.
 $ssh-keygen -t rsa
 $scp .ssh/id_rsa.pub hduser@bsw-data1/.ssh/authorized_keys
 $scp .ssh/id_rsa.pub hduser@bsw-data2/.ssh/authorized_keys
4. Open the file $HBASE_INSTALL_DIR/conf/hbase-env.sh and set the $JAVA_HOME.
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_25

5. Open the file $HBASE_INSTALL_DIR/conf/hbase-site.xml and add the following properties.
 
                
                                hbase-master
                                bsw-HbaseMaster:60000
                                The host and port that the HBase master runs at.
                               
                
                
                                hbase.rootdir
                                hdfs://bsw-HbaseMaster:9000/hadoop-datastore
                                The directory shared by region servers.
                
                
                                hbase.cluster.distributed
                                true
                                 Possible values are
                                false: standalone and pseudo-distributed setups with managed
                                Zookeeper true: fully-distributed with unmanaged Zookeeper
                                Quorum (see hbase-env.sh)
                                
                
                
                                hbase.zookeeper.property.clientPort
                                2181
                
                
                                hbase.zookeeper.quorum
                                bsw-HbaseMaster
                
 
Note:-
In our Example, Zookeeper and hbase master both are running in same machine.
6. Open the file $HBASE_INSTALL_DIR/conf/hbase-env.sh and uncomment the following line:
 export HBASE_MANAGES_ZK=true

7. Open the file $HBASE_INSTALL_DIR/conf/regionservers and add all the regionserver machine names.
   bsw-data1
   bsw-data2
   bsw-HbaseMaster
Note: Add bsw-HbaseMaster machine name only if you are running a regionserver on bsw-HbaseMaster machine.

    INSTALLING AND CONFIGURING HBASE REGIONSERVER

1. Download hbase-1.1.2tar.gz from http://www.apache.org/dyn/closer.cgi/hbase/ and extract it in some directory in your computer. Now this path is called as $HBASE_INSTALL_DIR.
2. Edit the file /etc/hosts on the hbase-regionserver machine and add the following lines.
 192.168.35.16 bsw-HbaseMaster       bsw-HbaseMaster
Note: In my case, bsw-HbaseMaster and hadoop-namenode are running on same machine.
Note: Run the command “ping bsw-HbaseMaster”. This command is run to check whether the bsw-HbaseMaster machine IP is being resolved to actual IP not localhost IP.

3.We have needed to configure password less login from bsw-data1 and bsw-data2 to bsw-HbaseMaster machine.
                 Execute the following commands on bsw-data1 and bsw-data2 machine.
$ssh-keygen -t rsa
$scp .ssh/id_rsa.pub hduser@bsw-HbaseMaster/.ssh/authorized_keys2

4. Open the file $HBASE_INSTALL_DIR/conf/hbase-env.sh and set the $JAVA_HOME.
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_25
Note:  If you are using open jdk , then give the path of open jdk.

5. Open the file $HBASE_INSTALL_DIR/conf/hbase-site.xml and add the following properties.

                
                                hbase-master
                                bsw-HbaseMaster:60000
                                The host and port that the HBase master runs at.
                               
                
                
                                hbase.rootdir
                                hdfs://bsw-HbaseMaster:9000/hadoop-datastore
                                The directory shared by region servers.
                

                                hbase.cluster.distributed
                                true
                                 Possible values are
                                false: standalone and pseudo-distributed setups with managed
                                Zookeeper true: fully-distributed with unmanaged Zookeeper
                                Quorum (see hbase-env.sh)
                                
                
                
                                hbase.zookeeper.property.clientPort
                                2181
                  

                
                hbase.zookeeper.quorum
                bsw-HbaseMaster
             
    

6. Open the file $HBASE_INSTALL_DIR/conf/hbase-env.sh and uncomment the following line:
export HBASE_MANAGES_ZK=true

Note:-
Above steps is required on all the datanode in the hadoop cluster.

START AND STOP HBASE CLUSTER

1. Starting the Hbase Cluster:-
we have need to start the daemons only on the bsw-HbaseMaster machine, it will start the daemons in all regionserver machines. Execute the following  command to start the hbase cluster.
$HBASE_INSTALL_DIR/bin/start-hbase.sh
Note:-
           At this point, the following Java processes should run on hbase-master machine.
 hduser@bsw-HbaseMaster:$jps
               14143 Jps
               14007 HQuorumPeer
               14066 HMaster
and the following java processes should run on hbase-regionserver machine.
              23026 HRegionServer
              23171 Jps
2. Starting the hbase shell:-
$HBASE_INSTALL_DIR/bin/hbase shell
                HBase Shell; enter 'help' for list of supported commands.
                Version: 0.20.6, r965666, Mon Jul 19 16:54:48 PDT 203
                hbase(main):001:0>
                Now,create table in hbase.
hbase(main):001:0>create 't1','f1'
                0 row(s) in 1.2910 seconds
                hbase(main):002:0>
Note: – If table is created successfully, then everything is running fine.

3. Stoping the Hbase Cluster:-
    Execute the following command on hbase-master machine to stop the hbase cluster.
   $HBASE_INSTALL_DIR/bin/stop-hbase.sh

2016年1月26日星期二

build hadoop-2.7.1 from source code on Ubuntu-15.10

1. Download hadoop-2.7.1-src.tar.gz and untar it

2. Folloing BUILDING.txt to install dependencies

3. But instead of installing oracle-java7-installer (which Oracle already restricts), install oracle-java8-installer

4. Do not install libprotobuf-dev and protobuf-compiler from apt-get, as it will pull version 2.6.1, but this version of hadoop requires 2.5.0. Instead download protobuf-2.5.0 from web, and run protobuf_arm64_patch.sh (attached below) to patch it, then do './configure; make; make install; ldconfig'

5. Do 'cd hadoop-maven-plugins; mvn install' before building hadoop. This is required for building any hadoop modules (not just eclipse support), otherwise you will run into mvn plugin error

6. Run 'mvn clean install -DskipTests -Pdist -Pnative' to build hadoop, you should find hadoop-2.7.1 directory under hadoop-dist/target

7. Then  follow http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html for single node setup. Remember to modify hadoop-env.sh to set JAVA_HOME variable


Contents of protobuf_arm64_patch.sh:

cd protobuf-2.5.0/
wget https://gist.github.com/BennettSmith/7111094/raw/171695f70b102de2301f5b45d9e9ab3167b4a0e8/0001-Add-generic-GCC-support-for-atomic-operations.patch -O /tmp/0001-Add-generic-GCC-support-for-atomic-operations.patch
wget https://gist.github.com/BennettSmith/7111094/raw/a4e85ffc82af00ae7984020300db51a62110db48/0001-Add-generic-gcc-header-to-Makefile.am.patch -O /tmp/0001-Add-generic-gcc-header-to-Makefile.am.patch
patch -p1 < /tmp/0001-Add-generic-GCC-support-for-atomic-operations.patch
patch -p1 < /tmp/0001-Add-generic-gcc-header-to-Makefile.am.patch
rm /tmp/0001-Add-generic-GCC-support-for-atomic-operations.patch
rm /tmp/0001-Add-generic-gcc-header-to-Makefile.am.patch