You can make a difference in the Apple Support Community!

When you sign up with your Apple Account, you can provide valuable feedback to other community members by upvoting helpful replies and User Tips.

Gigantic Library split over two locations

Got a new MacBook Pro M3 in Dec 2023 that has been pretty reliable. We made the switch from a Seagate 16TB USB external drive that was constantly ejecting to a RAID set-up that has been stable. This happened

in March 2024. The majority of the library (~11TB) is on the RAID unit (~20TB), but all the music that has been imported since March 2024 is on the local HD. Music plays everything without issue. Is there a good way to fix this without a giant headache? The internal HD has gone from 1TB down to 350GB left, so it's a significant amount of albums.

Drag the "Music" folder from the local HD to the RAID unit and point each album over there? Delete the folders (there are a lot) and re-import them once library knows to look at the RAID again? We use all the music for work and don't want to lose days or weeks fixing this if there's an easier solution.


Should the media file that tells music where everything is be on the RAID or the laptop?


Thanks!

MacBook Pro 16″, macOS 14.5

Posted on Oct 4, 2024 8:37 AM

Reply
23 replies

Oct 4, 2024 1:59 PM in response to DHeckeler

Exactly, anything located outside the designated media folder is copied into it. Whatever is already there stays put. See Managing your Mac media libraries - Apple Community for some further background. The media folder would typically contain folders for each type of media, or some of those and all artist folders, depending on any previous history with different builds of iTunes. While manually moving things around on local HFS+/APFS drives should be OK I'd be wary of moving anything by hand on the RAID.


tt2

Oct 7, 2024 6:34 AM in response to DHeckeler

Ouch. I'd have skipped the renaming if asked. It probably renamed files according to the details set out in the iTunes Media Organization section of Managing your Mac media libraries - Apple Community, which in and of itself ought not to have had any impact on the content of the files. You can recover the state of the library from the Time Machine backup, but do you have a backup of the RAID as well?


tt2

Oct 4, 2024 11:05 AM in response to DHeckeler

Use Music > Preferences > Files to set the media folder location to the folder containing your content on the RAID drive. You may be prompted to consolidate to that location when you set it, or you can do so via File > Library > Organize Library > Consolidate Files. This will create copies of the files that are currently on the local HD and update the library such that it points to the new copies. Once the process is complete you can manually clean up the old media folder location on the internal drive.


There is an advantage to having the .musiclibrary package in the same folder as the media folder as the library can then be easily cloned to another drive for security. Reading the library from a RAID location may be less efficient however, so there may not be any benefit in going through the steps to rearrange the library into this shape.


tt2

Oct 12, 2024 1:18 PM in response to DHeckeler

DHeckeler wrote:

I've never done this before. What should I be looking for in the XML file? Thank you!


XML is a text file, and is readable. Each field in an XML file is tagged, and related fields (such as each playlist entry) are nested.


Here is a hunk of the XML of an exported playlist, showing part of one track within the playlist:

<key>6842</key>
		<dict>
			<key>Track ID</key><integer>6842</integer>
			<key>Name</key><string>Espacio Sideral</string>
			<key>Artist</key><string>Jesse &#38; Joy</string>
			<key>Album Artist</key><string>Jesse &#38; Joy</string>
			<key>Composer</key><string>Jesse Huerta &#38; Joy Huerta</string>
			... {stuff here deleted } ... 
			<key>Apple Music</key><true/>
			<key>Location</key><string>file:///Users/mrhoffman/Music/iTunes/iTunes%20Music/Apple%20Music/Compilations/Jesse%20&#38;%20Joy/03%20Espacio%20Sideral.m4p</string>
			<key>File Folder Count</key><integer>5</integer>
			<key>Library Folder Count</key><integer>1</integer>
		</dict>


Each track will have information containing the full file name of the file containing the music.


From a test XML created and exported here (from a consolidated music library, which means stuff isn't located all over), the file location info from the XML file looks like this:

<key>Location</key><string>file:///Users/mrhoffman/Music/iTunes/iTunes%20Music/Apple%20Music/Compilations/Jesse%20&#38;%20Joy/03%20Espacio%20Sideral.m4p</string>


This location is a standard URL, like those that work with a web browser. (This uses standard URL encoding too, which means the %20 is a space, and &#38 is an & ampersand character.)


In this example, the file:/// means the location is a file, a colon and three slashes immediately following means no host name is specified; that the referenced location is local. If you see entries with a host name between the second and third slashes, that particular track is not located locally. It's on a file server or host somewhere, and that could conceivably be the local host accessed remotely.


An example of a remote file specification for a file served by the host fileserver.example.com looks like this:

file://fileserver.example.com/Users/mrhoffman {rest of path here}


You can use a search tool in your preferred editor, or a command-line tool such as grep, to search for just the location entries from the file, which will make this all less to look at.


The following is an example of using the grep tool available in zsh and other command line shells to search for the location entries for the tracks included in the playlist XML file named Jesse y Joy.xml :

% grep -i ">location<" "/Users/mrhoffman/Documents/Jesse y Joy.xml"
... {stuff expurated} ...
			<key>Location</key><string>file:///Users/mrhoffman/Music/iTunes/iTunes%20Music/Apple%20Music/Compilations/Jesse%20&#38;%20Joy/03%20Espacio%20Sideral.m4p</string>
... {stuff expurated} ...
mrhoffman@cupertino ~ % 


There are quite probably better tools for managing enormous libraries too, or at least for reading and processing XML playlist files.


TL;DR: look for locations in the XML that don't include a string starting with file:///

Oct 9, 2024 9:38 PM in response to turingtest2

Restoring the iPhone got it to sync again. There is one playlist that is corrupted and is causing the sync issue...it's the big one that has 54,000+ songs on it. Is there away to fix a broken playlist? Rebuilding it from scratch is doable, but annoying. I was able to get about 1600 songs to sync and that's it.


Ever time we try to sync with that playlist attached we get the same error that pops up. I've included a picture. I'm not sure how to scan to see what device is "120" that keeps getting disconnected


Oct 6, 2024 6:03 AM in response to turingtest2

Thanks for your help getting started on this.


Pointed Music to the RAID drive. It asked if we wanted to rename all the files so that it would be easy to find. Clicked on "Yes" and it started the process. Took about 16 hours and there are no gray exclamation points which is great! Tested 10 songs and 3 played fine, the other 7 play but about 8 seconds in the sound garbled kind of like a modem used to during dial-up days and you can't listen to them. I closed Music to restart and it brings up a "Saving Music Library" window that has a bar that goes back and forth on it for about 10 hours now. I checked under Force Quit menu and it doesn't say Music is not responding so we just let it go until it's done? How long could that possibly take?


At no point did it ask us if we wanted to consolidate the library. The music did not transfer to the RAID when it asked about renaming the files. I will do that next if the program ever actually quits. We have a Time Machine back-up of the computer from right before we started this (it backs up on hourly basis?) so if we need to start over we can. Ideally we don't want to lose days to this.


Any more thoughts?

Oct 7, 2024 8:00 AM in response to turingtest2

I sure hope it didn't rename anything based on outside sources. Yes we have a back up of the RAID, we do Carbon Copy Cloner on it several times a week. Had a external drive fail about a decade ago and will never make that mistake again! We then back up the CCC external drive to another so there's two back-ups at all times.


Force quitting and restarting Music fixed the two issues above. I will try to consolidate the libraries in a few days. Need to get caught back up after losing two days to the switch.


Thanks again for helping!

Oct 7, 2024 9:36 AM in response to DHeckeler

I suspect you're going to have to roll back everything, but try holding down option+cmd as you launch Music which opens it in its safe mode and does some internal consistency checks. If you're using a .musiclibrary restored from backup hold down option as you launch Music and explicitly chose the restored file in case the app is actually opening a cached copy of the library.


tt2

Oct 10, 2024 12:29 PM in response to DHeckeler

That error would suggest that you have one or more items that your library believes are stored somewhere on your network, at a fixed IP address rather than a named device. Could that be the case? One possible approach is to uncheck all items in that playlist and sync once more, (syncing only checked tracks of course) then check tracks in blocks of say 4,000 at a time until you see the error again. Once the error crops up play with the selection until you can determine what track causes the problem.


tt2

Oct 10, 2024 2:11 PM in response to turingtest2

The music is only on the RAID or the internal HD. We've not moved the songs from the laptop to the RAID yet hoping to iron out these issues first.


I'm hoping we can find an older back-up of the phone on the computer so we can get the songs back on the phone while we rebuild the playlist. We can't roll back the music library file because we've added a bunch of albums to the library after.


Some of the songs on playlists that did sync are missing artwork, which is definitely a new glitch.

Oct 11, 2024 8:29 AM in response to Gooner2105

Gooner2105 wrote:

Sorry to Hijack thread, but how can I pick your brain about Music problems turingtest2? I've sent you a message yesterday to samsoft and was just wondering if you could help,
Thanx
Steve


Please start your own thread as mixed threads get confusing, please include actual details about “music problems”, and please link to the user that you wish to ping. If tt2 doesn’t see it, I’ll get them over to your new thread as I’ll see your new thread. This if, it’s not something others can answer.

Oct 11, 2024 8:34 AM in response to DHeckeler

Whatever was done with locating or relocating those files, those files are being accessed remotely.


Whether you added that linkage without realizing, or some corruption thoughtfully added that?


First step is rebuilding the playlist.


Second is reviewing the 54,000 music files involved. (I keep my music library consolidated, so I don’t know if Get Info (in Music app) will show a local or remote location for a song. It might.)

Gigantic Library split over two locations

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple Account.