Sign In

stealth software | blogs

 

  
Embry-Riddle Prescott and Stealth Software Partner to Benefit Students, Consumers
RBS and Content Database Limits in SharePoint 2010 / 2013
SharePoint 2013 - Data Shredding and Remote Blob Storage (RBS)

 

Blogtext
Blogger Information

Embry-Riddle Prescott and Stealth Software Partner to Benefit Students, Consumers

Written by Jason Kadah

​Embry-Riddle Aeronautical University and Stealth Software of Scottsdale, Ariz., have joined forces to create new applications and storage methods that will prevent the leakage of sensitive information in the commercial world as well as in government. The agreement will allow Embry-Riddle’s College of Security and Intelligence (CSI) students and university researchers to work with a leading software company, learning how to securely store and share critical data and files.“Our students will be able to use state-of-the-art solutions that they will see when entering the working world,” said Dr. Jon Haass, Associate Professor of Cyber Security and Intelligence at Embry-Riddle Prescott. “Experience with database solutions and secure applications make them more valuable to prospective employers.”

StealthSoftware.jpg

Chancellor Frank Ayers signs the partnership with Stealth Software.

Embry-Riddle’s CSI, the first of its kind in the country, is on the forefront of establishing one of the most advanced cyber security programs in the nation. Stealth Software provides software solutions to enhance the security, scalability and performance of Microsoft SharePoint (widely used by the U.S. government and the intelligence community) as well as .NET applications while reducing cost, complexity and management overhead. The company has developed a data security capability that is directly applicable in that environment and will now be used at the College of Security and Intelligence. Additionally, Stealth Software will now be implemented and utilized in the CSI cyber lab.
“With the creation of the College of Security and Intelligence, Embry-Riddle is on the forefront of establishing one of the most advanced programs in the nation for educating students in leading-edge cyber security technologies,” said Gerard Warrens, CEO of Stealth Software. “Bringing Stealth Software onto the campus and into the CSI and its cyber lab bolsters the student experience while preparing students for a career in the U.S. law enforcement, homeland security and intelligence community.”
The agreement will impact and ultimately benefit the public. Both Embry-Riddle and Stealth Software recognize the need to train more skilled cyber professionals in order to create a more secure and trusted cyber space. “The goal is for everyone to safely interact, shop, bank and enjoy the many benefits of the globally connected Internet,” Haass said.
“Stealth Software will have the benefit of partnering and supporting one of the most prestigious universities in the country and supporting the national security mission,” added Warrens.
 
About Stealth Software
 
Stealth Software is a leading and award winning provider of privacy, security and compliance solutions for both on premise and cloud environments for government and commercial clients with offices in Scottsdale, Ariz. and Luxembourg.  Stealth Software has patented, software-only solutions to enhance the scalability, performance and security of Microsoft SharePoint while reducing costs, complexity and management overhead. The company strongly believes in hands on experience when it comes to preparing the Cyber Security and Intelligence workforce of the future.  For more information, visit http://www.stealth-soft.com.
 
About Embry-Riddle
 
Embry-Riddle Aeronautical University, the world’s largest, fully accredited university specializing in aviation and aerospace, is a nonprofit, independent institution offering more than 70 baccalaureate, master’s and Ph.D. degree programs in its colleges of Arts & Sciences, Aviation, Business, Engineering and Security & Intelligence. Embry-Riddle educates students at residential campuses in Daytona Beach, Fla., and Prescott, Ariz., through the Worldwide Campus with more than 150 locations in the United States, Europe, Asia and the Middle East, and through online programs. The university is a major research center, seeking solutions to real-world problems in partnership with the aerospace industry, other universities and government agencies. For more information, visit http://www.embryriddle.edu, follow us on Twitter (@EmbryRiddle) and facebook.com/EmbryRiddleUniversity, and find expert videos at YouTube.com/EmbryRiddleUniv.

 

RBS and Content Database Limits in SharePoint 2010 / 2013

The Defined Limits
SharePoint has a stated content database size limit of 200 GB or 4TB (if required disk sub-system performance can be achieved).

This limit is described as a 'Supported' limit by Microsoft which means "Configurable limits that have been set by default to a tested value". This is the lowest of the three different types of limits set by MS, the others being Thresholds - "Configurable limits that can be exceeded to accommodate specific requirements" - and Boundaries – "Static limits that cannot be exceeded by design".

To further illustrate Microsoft's stance on Supported limits – "It is important to be aware that many of the limit values that are provided in this document represent a point in a curve that describes an increasing resource load and concomitant decrease in performance as the value increases. Therefore, exceeding certain limits, such as the number of site collections per web application, may only result in a fractional decrease in farm performance." Although they do then note that exceeding such limits "is not a best practice".

How does this affect RBS?
Microsoft state within their discussion on the 200GB limit that "If you are using Remote BLOB Storage (RBS), the total volume of remote BLOB storage and metadata in the content database must not exceed this limit." Given that RBS improves overall SharePoint performance and reduces load on the content database's disk sub-system it would seem to be merely a documentation oversight that RBS is not mentioned in the supported limit of 4TB.

Microsoft state that "thresholds and supported limits guidelines are determined by performance", but in the case of RBS they do not test with third party providers such as Stealth Content Store for SharePoint (SCSS). They acknowledge the increase in performance and reduction in SQL load, they state that RBS is of optimal use in situations where (amongst other things) "The resources on the computer that is running SQL Server might become a performance bottleneck" [technet].

Summary
This leads to an interesting outcome. Microsoft has defined a Supported limit that is based on internal performance testing. In the case of RBS this is without access to the actual software used to implement this feature.

By their own acknowledgment the Supported limits may be exceeded based on specific requirements and result in minimal performance degradation. In the case of SCSS you are already getting a performance increase that exceeds the baseline.

If operational requirements create a need for exceeding the 200GB limit then that is the overriding business requirement. A Supported limit will not prevent that, the performance should be monitored and certainly an RBS provider such as Stealth Content Store for SharePoint will help mitigate any problems with SharePoint's performance.

All quoted sections are taken from http://technet.microsoft.com/en-us/library/cc262787.aspx unless otherwise stated.

Sharepoint 2013 – Data Shredding and Remote Blob Storage (RBS)

Data Shredding Overview
Data Shredding is a new feature of SharePoint 2013, it is enabled by default. Its purpose is to reduce disk IO for updates to documents. It is particularly targeted at modern MS document formats (docx, xlsx etc) and has mixed results on other document types (particularly compressed data, but also pdfs and older MS formats).

The best case scenario for Data Shredding is medium to large modern MS documents that are frequently updated with versioning turned on. The initial version of a file will take up slightly more space on disk (and thus require slightly greater disk IO), but subsequent updates will only update modified chunks which leads to a long term reduction in both disk IO and total storage. The expected tipping point for a reduction in overall storage size is between 2 and 3 revisions of a document. It is also worth noting that without versioning the overall storage utilisation will always be higher, but the disk IO still will be lower.

http://www.sharepointpromag.com/blog/dan-holmes-viewpoint-on-sharepoint-blog-24/sharepoint-2013/sharepoint-2013-shredded-storage-144987

Shredded Files
The RBS interface has not been changed between SharePoint 2010 and SharePoint 2013. This means an RBS provider such as Stealth Content Store for SharePoint will still externalise the data provided to it as before. As we saw earlier though the data will now be shredded.

Seeing the impact of shredding on data stored within the database is fairly difficult, but when we inspect the external data storage utilised by RBS the impact of shredding becomes more obvious. The following images show a simple text file (Figure 1) that was uploaded to SharePoint 2013 and the resultant externalised blob (Figure 2).

Figure 1 A simple text file (40 bytes)

Figure 2 The resultant blob (1,251 bytes)

If you look carefully you can see the original data (towards the end of the second line of text), but in this case there is also a large amount of extra data.

In larger files you would expect to see multiple shredded blobs (Figure 3)

Figure 3 A Shredded document

Dependent on file size and FileWriteChunkSize you can expect to see blobs containing: User Data, Cobalt metadata or a combination of the two.

http://blogs.technet.com/b/wbaer/archive/2012/11/12/introduction-to-shredded-storage-in-sharepoint-2013.aspx

Recommendations
So, is Data Shredding worth it?
Well, the jury's still out on this one. If you have versioning enabled and use SharePoint primarily for the storage of your modern MS office documents then it's a definite yes. If you use SharePoint as a file repository for commonly read but rarely modified documents then maybe not.

It is a matter of balancing your number of updates, preference for MS document formats and average file size to find out whether it is worthwhile. Such metrics would need to be taken on a live environment and would require many weeks of minor alterations followed by further testing to fine tune perfectly. Chances are though that an investigation of that magnitude is not worth the time it would take.

Microsoft's recommendation is obvious – they enable shredding by default. Unless your data clearly sits outside the type of files Data Shredding was created for there is no reason not to leave it on.

Doesn't RBS conflict with Data Shredding?
Not really, the two technologies were created to solve completely different problems. RBS externalises data to reduce Content Database size (by 80-95%) and increase SharePoint performance with larger files. Data Shredding reduces disk IO and in some cases has a small reduction in Content Database size.

The two technologies do come into a small amount of conflict though. Although at Stealth we see read/write performance improvements from sub 100kb files the greatest improvements come as the blobs gets bigger. So, RBS likes larger blobs but Data Shredding creates smaller blobs. Fortunately there is the option to alter size of the Shredded chunks using the FileWriteChunkSize, the sweet spot is between 100kb and 2mb and depends again on the number of versions and average file size. More versions means a lower preference, larger files means we move towards the upper end of the range.