BBFS - Backup system, general workflow: Content server monitors a set of local directories. The files in those directories are indexed (calculating their SHA1). The indexed SHA1 (content hash) is stored in a file called content data. When requested content data is sent to backup server. When requested , files are sent to backup server for backup. Backup server requests new/changed contents (files) from the Content server. Backup files (copied from content server) are stored in a destination dir which is also monitored, indexed and stored in a file (backup content data). The destination dir structure is: backup/destination/tmp - used to temporary files while copying from content server. backup/destination/remote/ - content data received from servers. backup/destination/* - rest of dirs used to stored backup files. Each period of time (configurable), sends request to content server to get its latest content data. Each period of time (configurable), performs a sync check between content server and itself. If needed, requests and copy required files from content server to backup (store in destination dir). Installing: Install Ruby 2.1.1 Instal RubyGems Install application gems: gem install content_server (This can take a while till all gems are installed.) Running: 1. Preparing content server and backup server configuration files (see Appendix examples). 2. Run content server: content_server --conf_file=path_to_configuration_file 3. Run backup server: backup_server --conf_file=path_to_configuration_file Uninstalling all gems Unix: for x in gem list --no-versions; do gem uninstall $x -a -x -I; done Windows: ruby -e "gem list.split(/$/).each { |line| puts gem uninstall -Iax #{line.split(' ')[0]} unless line.empty? }" Note: Be aware to use it when some other programs run on ruby and use different gems. If this is a situation remove gems manually and don’t remove ruby. Validation: This release was manually tested. The test included backing up content server during a period of one week. During the week, files were generated randomly on content server machine. 14K content instances (files) were generated, which actually are 5K different contents of total size of 72MB to backup servers. The test was performed on 2 linux machines. During validation each 6 hours, backup server compared successfully files against the expected files. Documentation: Can be found at: https://github.com/bbfsdev/bbfs/wiki/Documentation Appendix (configuration files): Notes: User needs to change dummy values in the file example for parameters: monitoring_paths, from_email, and to_email. User can change other parameter according to the system characteristics (files size to backup, network speed etc..). Pls see more on this in documentation. 2. backup_server application configuration file (example): Notes: User needs to change dummy values in the file example for parameters: monitoring_paths, from_email, and to_email. User can change other parameter according to the system characteristics (files size to backup, network speed etc..). Pls see more on this in documentation.
Project
process_monitoring
Holds variable set which is regularly updated and export it to http protocol
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
Pull Requests
Development
Project Readme