SharePoint | Leveraging data management options
In SharePoint, there are many options and APIs to move, copy, duplicate, export content from one place to the other.
Common scenarios involve performing tasks such as archiving, back-ups, or related actions that require fetching information from a source and sending it over to a destination.
This post enumerates a few of those options, along with their benefits and caveats.
One of the most common means of backing up and/or migrating content is using the whole database.
Databases store pretty much all the content of one or more sites, and they are our best option for reproducing content, while maintaining all the integrity of the source.
Backups can be created by copying the physical .mdf and .log.mdf files, or by exporting the database into a .bak file.
This is a brief example procedure for backing up a content database:
Context: Single Site Collection, Multiple Web Sites(.bak)
This CmdLet enables us to export a whole site collection, maintaining its GUIDS.
"It preserves the GUID of every object except the GUID of the Site: when you restore the backup, SharePoint generates a new GUID for the site collection."
Restore-SPSite | Backup-SPSite
This CmdLet enables us to export a web site, or parts of a web site, such as a list.
There are some caveats to this approach, specially if you are looking for an exact replication of a site (e.g. reproduce a production environment).
"A major drawback of this operation is that it does not preserves workflows instances, associations, history and tasks."
Export-SPWeb | Import-SPWeb
This is a useful CmdLet for moving a site collection from one content database to another.
Move-SPSite | Copy-SPSite
This method is able to copy an item from a source list to a destination list.
It is meant for usage in Document Libraries, although it has been proven to work with list items as well (see link at the end).
SPFile.CopyTo | SPFile.MoveTo
This is one of the best alternatives to CopyTo/MoveTo, as they don't always work, specially on lists with many custom fields and content types.
Adding a new item based on an existing item proved to be a good workaround when copying/moving items.
SPFileCollection.Add
SendTo is a simple, yet functional way of moving items through the UI.
When we move an item, we have the option to keep it linked to the original item or create a new copy.
This is by far one of my favourites.
Although the option to open a list in explorer is only available in Internet Explorer, we can also map a folder to take advantage of this easily.
One of the best cases most people are unaware is that we can effectivelly move a document, while keeping all its metadata and versions, just as long as we actually "move" it, not "copy" it.
This may not be true for certain file types, such as PDFs.
Another nice perk is being able to hide specific files or folders, based on the windows "show hidden files and folders" setting.
SharePoint does this by default on items with their names start with an underscore.
This option is available under Site Actions, "Manage Content and Structure" in the UI.
It's not the most famous feature ever, but for some cases, it serves its purpose, which is to copy or move items within the same site collection.
Templates in SharePoint (.stp, .wsp) are great ways of exporting content. They seem like a nice approach, but quickly we start identitying a few caveats:
Nice open source tool from Codeplex.
"The SharePoint Content Deployment Wizard provides the means to deploy the following content:
- site collections
- webs
- lists
- folders
- list items (including files/documents)"
Content is exported using the Content Migration API as a .cmp file"
AvePoint's DocAve, Metalogix's Content Matrix, Kalmstrom's SP Archive, and others, are powerful tools and enable doing a lot of the above things without any code, but they are often expensive and they are not always suitable for every task, specially smaller operations.
Sources
Difference between STSADM Export and Backup,
http://blogs.msdn.com/b/yvan_duhamel/archive/2009/05/18/some-key-differences-between-stsadm-export-and-backup-operations.aspx
http://sharepoint.stackexchange.com/questions/62152/sharepoint-2010-what-is-the-difference-between-backing-up-a-site-collection-and/62156#62156
Content Deployment Wizard,
http://spdeploymentwizard.codeplex.com/
Moving list items between folders,
http://sharepointstruggle.blogspot.se/2010/07/sharepoint-vs-powershell-moving-list.html
SPFile.CopyTo VS RootFolder.Files.Add
http://sharepoint.stackexchange.com/questions/78426/spfile-copyto-vs-rootfolder-files-add
How can you move a document with its version history to a new library?
http://sharepoint.stackexchange.com/questions/1278/how-can-you-move-a-document-with-its-version-history-to-a-new-library/1507#1507
Common scenarios involve performing tasks such as archiving, back-ups, or related actions that require fetching information from a source and sending it over to a destination.
This post enumerates a few of those options, along with their benefits and caveats.
Content Database
Context: Multiple Site Collections (.bak)One of the most common means of backing up and/or migrating content is using the whole database.
Databases store pretty much all the content of one or more sites, and they are our best option for reproducing content, while maintaining all the integrity of the source.
Backups can be created by copying the physical .mdf and .log.mdf files, or by exporting the database into a .bak file.
This is a brief example procedure for backing up a content database:
- In SQL Server Management Studio, right-click the content database, choose Tasks, Backup
- Make sure to choose "Full" backup and "Add" a storage location
- In SQL Server Management Studio, create a new, empty database with a name of choice
- Right-click the new database and pick Tasks, Restore, Database
- Click "From device" and pick the .bak file
- Enable the "Restore" checkbox
- Go to Options and enable "Overwrite the exiting database (WITH RELPACE) so that you overwrite the newly created database with the backup database
- In a SharePoint Management Shell
- Dismount-SPContentDatabase [OldDatabaseName]
- Mount-SPContentDatabase [NewDatabaseName] -DatabaseServer [ServerName] -WebApplication [WebUrl]
Backup-SPSite / Restore-SPSite
Context: Single Site Collection, Multiple Web Sites(.bak)
This CmdLet enables us to export a whole site collection, maintaining its GUIDS.
"It preserves the GUID of every object except the GUID of the Site: when you restore the backup, SharePoint generates a new GUID for the site collection."
Restore-SPSite | Backup-SPSite
Export-SPWeb / Import-SPWeb
Context: Single Web Site or specific Web resources (e.g. List) (.cmp)This CmdLet enables us to export a web site, or parts of a web site, such as a list.
There are some caveats to this approach, specially if you are looking for an exact replication of a site (e.g. reproduce a production environment).
"A major drawback of this operation is that it does not preserves workflows instances, associations, history and tasks."
Export-SPWeb | Import-SPWeb
Move-SPSite / Copy-SPSite
Context: Single Site CollectionThis is a useful CmdLet for moving a site collection from one content database to another.
Move-SPSite | Copy-SPSite
SPFile.CopyTo / SPFile.MoveTo
Context: Single Item (List/Library)This method is able to copy an item from a source list to a destination list.
It is meant for usage in Document Libraries, although it has been proven to work with list items as well (see link at the end).
SPFile.CopyTo | SPFile.MoveTo
RootFolder.Files.Add()
Context: Single Document Library ItemThis is one of the best alternatives to CopyTo/MoveTo, as they don't always work, specially on lists with many custom fields and content types.
Adding a new item based on an existing item proved to be a good workaround when copying/moving items.
SPFileCollection.Add
SendTo
Context: Single Item (List/Library)SendTo is a simple, yet functional way of moving items through the UI.
When we move an item, we have the option to keep it linked to the original item or create a new copy.
File Explorer
Context: Multiple Document Library ItemThis is by far one of my favourites.
Although the option to open a list in explorer is only available in Internet Explorer, we can also map a folder to take advantage of this easily.
One of the best cases most people are unaware is that we can effectivelly move a document, while keeping all its metadata and versions, just as long as we actually "move" it, not "copy" it.
This may not be true for certain file types, such as PDFs.
Another nice perk is being able to hide specific files or folders, based on the windows "show hidden files and folders" setting.
SharePoint does this by default on items with their names start with an underscore.
$folder = (Get-SPWeb http://url).Folders["DocLib_Name"] $folder.Properties["vti_winfileattribs"]="00000016" $folder.Update() $file = (Get-SPWeb http://url).GetFile("/doclib/file.extension") $file.Properties["vti_winfileattribs"]="00000016" $file.Update()
Site Content and Structure
Context: Site Collection / Multiple Document Library ItemThis option is available under Site Actions, "Manage Content and Structure" in the UI.
It's not the most famous feature ever, but for some cases, it serves its purpose, which is to copy or move items within the same site collection.
Templates (Web, List)
Context: Single Web Site or Single List (.stp)Templates in SharePoint (.stp, .wsp) are great ways of exporting content. They seem like a nice approach, but quickly we start identitying a few caveats:
- Limited (customizable) content size
"Site templates are saved from within a site as an .stp file that contains a set of all modified files and lists in the source site. The file can also contain content from the source site, such as list items and documents. By default, the maximum size of an .stp file is 10MB." [ref]
Workaround: bump up to 100MB (maximum is 500)
stsadm -o setproperty -propertyname max-template-document-size -propertyvalue 100000000
or PowerShell [ref]
$webservice = [Microsoft.SharePoint.Administration.SPWebService]::ContentService $webservice.MaxTemplateDocumentSize = [new_value] $webservice.Update()
- Only for Webs, not Sites
Workaround: save each site individually and restore them individually
- Dependencies
One of the biggest challenges is dependencies.
Whenever we save a template of a site, it stores references of the web and site scoped features.
For lists, we also must ensure the same columns and content types exist, otherwise, we may be able to create the templates, but we won't be able to create lists based on that template later.
Links:
_layouts/savetmpl.aspx (available through the UI once the publishing feature is enabled)
Example: Creating a new web based on a web template through PowerShell
#see which templates are available $site = Get-SPSite http://weburl/sites/site1 $templateName = $site.GetWebTemplates(1033) | where-object{$_.Title -eq "my_template_title"} | select Name #"{CCC38530-B9B3-418A-BB87-91CA7F626711}#my_template_title" #create a new web based on a template $web = New-SPWeb http://weburl/sites/site1/web1 -Name mynewweb -UseParentTopNav -Language 1033 $web.ApplyWebTemplate($templateName)
Content Deployment Wizard
Context: Site Collection / Web Site / ListNice open source tool from Codeplex.
- site collections
- webs
- lists
- folders
- list items (including files/documents)"
Content is exported using the Content Migration API as a .cmp file"
3rd Party Tools
Context: ManyAvePoint's DocAve, Metalogix's Content Matrix, Kalmstrom's SP Archive, and others, are powerful tools and enable doing a lot of the above things without any code, but they are often expensive and they are not always suitable for every task, specially smaller operations.
Sources
Difference between STSADM Export and Backup,
http://blogs.msdn.com/b/yvan_duhamel/archive/2009/05/18/some-key-differences-between-stsadm-export-and-backup-operations.aspx
http://sharepoint.stackexchange.com/questions/62152/sharepoint-2010-what-is-the-difference-between-backing-up-a-site-collection-and/62156#62156
Content Deployment Wizard,
http://spdeploymentwizard.codeplex.com/
Moving list items between folders,
http://sharepointstruggle.blogspot.se/2010/07/sharepoint-vs-powershell-moving-list.html
SPFile.CopyTo VS RootFolder.Files.Add
http://sharepoint.stackexchange.com/questions/78426/spfile-copyto-vs-rootfolder-files-add
How can you move a document with its version history to a new library?
http://sharepoint.stackexchange.com/questions/1278/how-can-you-move-a-document-with-its-version-history-to-a-new-library/1507#1507
I feel very grateful that I read this. It is very helpful and very informative and I really learned a lot from it. Learnerships
ReplyDeleteDocument Management Server
ReplyDeleteDocument Management Server is a professional document scanning company.Who provide Genuine Database Registered Documents, High quality register Documents
https://documentmanagementserver.net/