Memory Used doesn't match Total Memory for all Process

Cached Files: 25,34 GB

It is faster and MUCH more power efficient to access a file the 2nd, 3rd, 4th, … time if it is cashed in memory, then it is to read it from storage.

If there is no more free memory, and macOS needs the memory for an app, it will take it from the cache.

If the cached data already has an identical copy on disk, macOS only needs to update in memory tables to move the memory from the cache to the app.

If the cached data is “Dirty” (has modified data, not already written to storage), macOS will first write that data to storage, then give it to an app.

macOS will periodically flush dirty cache data to storage, so cached data is mostly just an in memory copy of data on the storage device.

Also macOS prioritizes choosing clean cached data over dirty cached data when giving the memory to an app.

In your case, you would have to first use the 33,73GB of free memory, before macOS would start to take away the cached file memory.

PS. I have maintained and developed new features on Unix based file systems for several Fortune 500 companies since ‘95. However, I have never worked for Apple, and have never looked at the macOS source code, so some small details may be wrong.

ccarvalho@apple wrote:

I've a new Mac and I find something really Bizarre. In My Old Mac, an Mac Pro, Intel, 16G Ram, Memory used with quite a few applications running was around 12G. This was really a Lab, believe, I've worked with everything including everything in this laptop and still working, but it's outaded for my current needs.

Now on my new MAC with 64GB, I did a clean install, and I've nothing run on it, except Chrome, one excel file and I've 32GB of memory Used? What the hack

When I add up all of the memory columns from your spreadsheet, I come out to 10G. I'm assuming there are lots of processes off screen that take up less memory. So if this is how you were measuring memory before, the usage seems about identical.

When I check activity monitor, and I make the addition of memory used for all process running, I can't reach 32GB, no way, so, where it comes from ? Is there a memory management issue with mac or is it a bug with the activity Monitor?

Memory is very complicated these days. There are many different types of memory, many different uses, many different perspectives. Are you noticing any kind of performance problem? If so, that's what you need to focus on. Don't go looking for problems. I've seen many people go looking for problems, find a few hundred thousand of these confusing details and totally lose their minds to paranoia. I'm not exaggerating here.

I will indulge your quest for information just a little bit. Hopefully it won't cost you your sanity this time. How are you measuring this memory usage? What does that "Mem..." column in your list mean? (It literally doesn't even say "Memory".) Now, control click or right-click in the column header row. How many other potential columns are there? How many of them are "Memory"?

To be honest, Apple's display of these things leaves a lot to be desired. Your new Mac is literally just a souped-up iPhone with a physical keyboard and larger screen. And there's been a generation of Apple engineers who have only ever known iPhone development, where none of this information is available. They simply don't know what the screen is supposed to show. At this point, on the Mac, it's little more than pretty lights. The numbers simply aren't ever going to add up. Today, you're confused about memory. I can assure you that you do not want to turn your attention to storage. And don't try to get clever and bring up a help page either.

I assume you bought this computer for some specific reason? Was that reason to learn about modern memory management strategies on a 40 year-old hybrid-OS ported back and forth across 5 different architectures? I'm guessing no. So use the device for what you bough it for. If you encounter any problems, come back and ask here. Don't do internet research. The internet is wrong.

ccarvalho@apple wrote:

Juts to give some details regarding why I bought new, one, I do lots of Machine Learning, and I use GPU quite often, and Memory is also critical.

Remember that it's a consumer machine. There is a significantly large layer of system controls and information that is little more than pretty pictures. This has turned into a significant problem in Sequoia. It turns out there are lots of people who had carefully configured all those pretty apps "just to be safe" and then installed a layer of scam ware on top. Those people are really suffering right now. Once you start doing professional work, you have to remember the intended audience of your tools.

In your case, given your description of what you're doing, that "Java" process is what most concerns me. As you know modern machine learning is heavily memory and GPU dependent. It's very unlikely that any Java app running on a Mac is going to properly handle either memory or GPU. You'll need specialized tools built for Apple's Metal framework. Some of the popular open source and/or OpenGL/CL/etc. things might work, but will likely require significant configuration and very careful management. You can expect them to break with each and every OS update.

The operating system is going to optimize the memory management based on what you are doing. Based on your screenshot, your machine learning tools are well into the noise. Your computer is running Java and Chrome. The system is trying to keep those two apps happy. They will be using all of your RAM and your GPU. Your DataScienceStudio apps will have to get by on whatever is left over.

It is not just application code, but any file read and or written by the Applications, agents and daemons that are running on your Mac.

Also the memory used as reported by Activity Monitor, will include shared libraries, frameworks and such which may be reported in multiple apps, agents and daemons.

Understanding modern operating system virtual memory management is full semester college course. And that would only cover a single operating system (most likely Linux) and not Mac, Windows, Solaris, AIX, etc…

If you have a performance issue, that you think is memory related, then you might want to dig into what is using your memory. As it is you have tons of memory you paid for that is being wasted, because it is not being used.

When it comes to memory vs storage, a good operating system will fill up memory and have a small amount of free memory, so it is fast and power efficient. Any time it has to get something from storage or the network, it is both slower and uses more electricity.

Your memory is not being "eaten".

Your memory is being USED.

Unused memory just sits there.

Your mac can respond much faster with material that is cached in memory than stuff that is retrieved from disk (even if it is an SSD, it is still orders of magnitude slower).

This does not mean that if you have an app that needs 32GB of RAM or whatnot, it cannot get it. Things are working as designed.