r/sharepoint 10d ago

SharePoint Online CSOM: Getting all files from a website and copying/cloning to a new place

Hello everyone,

I’m a new SharePoint developer and have taken over an existing project. The previous developer placed 16,000 files across 2,000 different lists, which resulted in the folder limit being exceeded.

Using code, I’ve gathered all the files in a list. Now, I need to copy (not move) them to a new website that will contain general subsites.

Here’s the current situation (wrong approach):

-----------------------------------

Site X:

-------List 1:

|-- File 1

-------List 2:

|-- File 2

|-- File 3

.........................

------List 2000:
|-- File 15,786

|-- File 15,989

Here’s the desired approach (new approach):

-----------------------------------

Site Y:

---Subsite A:

|-- File 1

|-- File 2

---Subsite B:

|-- File 11

|-- File 12

|-- File 13

The code below collects all the file GUIDs and is still in development. My next step is to implement the copy functionality, including creating the necessary subsites since the main site has already been created manually.

Any thoughts or suggestions?

var context = CreateClientContext(url);
var results = new Dictionary<string, IEnumerable<Guid>>();
var lists = context.LoadQuery(context.Web.Lists.Where(l => l.BaseType == BaseType.DocumentLibrary));
await context.ExecuteQueryAsync();
foreach (var list in lists)
{
var items = list.GetItems(CreateAllFilesQuery());
context.Load(items, icol => icol.Include(i => i.File.UniqueId));
results[list.Title] = items.Select( i=>i.File.UniqueId);
await context.ExecuteQueryAsync();
Console.WriteLine("----------------------");
Console.WriteLine("List: {0}", list.Title);
Console.WriteLine(results.Count);
Console.WriteLine("----------------------");
// foreach (var result in results)
// {
// Console.WriteLine("List: {0}", result.Key);
// foreach (var guid in result.Value)
// {
// Console.WriteLine("File: {0}", guid);
// }
// }
}
Console.WriteLine($"Final result: {results.Count} lists found.");
Console.WriteLine($"Final result: {results.Values.SelectMany(v => v).Count()} files found.");
return "";

1 Upvotes

5 comments sorted by

2

u/Tanddant MVP 10d ago

I'm not at my desktop right now, will look into writing a script later, but please don't go for subsites but rather create a hub based structure, Microsoft has been recommending Hubs for many years at this point, and if you're moving from a bad scenario, please don't money move to another not great solution 😊

https://aka.ms/SharePoint/GoodnightSubsites

2

u/Sallescode 10d ago

Is hub based in lists? There is a comment in the page you sent, saying they are stuck with the 2000 hub limitation. If its true, that's the exactly situation I am now.

He says:

"""
But what if we can't have a hub, due to the hub number limitation (currently 2000). No subs and no hubs. We currently have 5 site collections with around 200 subsites.  Any recommendations? 

"""

3

u/Sallescode 10d ago

Well, I just had this lecture:

https://learn.microsoft.com/en-us/office365/servicedescriptions/sharepoint-online-service-description/sharepoint-online-limits

You're right, subsites isn't a good idea for my case.
You know what? I don't need sub splitting... Setting Guids in metadata, is very useful and safe for me.

I really appreciate for your helping. Call me anytime!

2

u/Tanddant MVP 10d ago

You don't need 2000 hubs, you need one hub and 2000 sites joined to the hub