In June 2021, we announced several important updates for OneDrive on macOS, including an update to our Files On-Demand experience. This new experience is better integrated with macOS and will also help enable new features like Known Folder Move for macOS.
Today, we are excited to share that we have begun rolling out the new Files On-Demand experience to all our customers using macOS 12.1 or later. We also want to share some additional details about how the new Files On-Demand experience works, what changes you can expect, and when you can expect them.
We know that many of you are supporting organizations with lots of Macs that run OneDrive, and the more information we can provide you, the better you can serve your users.
Why we are building a new experience
In 2018, we shipped the first version of Files On-Demand for macOS. Since then, we’ve rolled it out for everyone using macOS 10.14 (Mojave) or later.
We are building a new experience for several reasons. One of the most important is that the new technology stack (based on Apple’s File Provider platform) is much better integrated with the operating system compared to the first version. This means a better user experience, better application compatibility, and better reliability. This technology stack also enables us to offer new features that we couldn’t offer before, like Known Folder Move, along with lots of other little improvements that you probably won’t notice right away!
Because the new experience is more integrated with macOS, it will have long-term support from Apple. The first version of Files On-Demand is built on several pieces of technology that are now deprecated. Moving to the new platform enables us to support this feature for years to come.
Supported macOS versions
For users who are not part of our Insiders program, the new Files On-Demand experience requires macOS 12.1 or later. Users on our Insiders program can continue to use macOS 12.0, but we strongly encourage them to update to the latest version.
macOS 12.2 will be the last version that supports the classic Files On-Demand experience. For future macOS versions, this means:
- Files On-Demand will default to on for all users and cannot be disabled.
- Devices will migrate automatically to the new Files On-Demand as soon as they receive a macOS update. You cannot delay this update without also delaying an update to macOS.
- Both our Standalone and App Store versions of OneDrive will have the same behavior.
- Users running a developer or beta version of macOS will have the same experience as a release version of macOS.
If you are concerned about application compatibility with this change, we strongly suggest that you install macOS 12.1 or later and test your workload. Depending on the results of that test, you may want to delay updating your Macs.
File system requirements
The new Files On-Demand experience requires a volume that is formatted with APFS. HFS+ volumes are not supported.
We are ending support for HFS+ after macOS 12.2. A very small minority of customers are syncing OneDrive on HFS+ volumes today. As we roll out the new Files On-Demand, these users will first experience a warning in the OneDrive activity center telling them to upgrade to APFS. Once the new Files On-Demand experience is fully rolled out, OneDrive will not launch until the volume is upgraded to APFS.
You can use Disk Utility (built in to macOS) to determine which volumes are running APFS, and upgrade any HFS+ volumes to APFS.
Sync root location
When users set up OneDrive, they must choose a location where files will sync. This is called a sync root. Historically, we’ve allowed users to choose any location on any fixed volume mounted on their Mac.
With the new Files On-Demand experience, the sync root is always located within users’ home directory, in a path such as:
~/Library/CloudStorage/OneDrive-Personal
As part of the upgrade, the sync root will be moved to this location. This location cannot be moved or changed and is controlled by macOS.
This path is a little cumbersome for users to use, so they can access this directory in two other ways:
- Under Locations on the Finder’s left navigation pane.
- Via a symlink at the original location the user picked when setting up OneDrive. For example, if the user chose to sync OneDrive at ~/OneDrive, then a symlink will be created from here to ~/Library/CloudStorage/OneDrive-Personal.
Cache path
To support the new experience, OneDrive maintains a cache path in a hidden location. This path contains a replica of the file tree that the user is syncing. Most of the files in this location are usually dataless and don’t consume disk space, but occasionally files here can have data, such as if a user pins a file or if a change is being transferred to or from the cloud.
OneDrive tries to maintain as little data here as possible, and instead prefers to keep data in the sync root. As such, file data is not generally kept in both locations unless a file is marked as “Always Available on This Device.” In that case, the file’s data will sometimes be retained in both the sync root and the cache, but the files will be linked using a clone, so they do not occupy any additional space.
Using another volume
Sometimes, users choose a path on another volume to set up OneDrive. A typical use case for this happens when a user has a small internal drive on their Mac, but also has a larger external drive attached.
This configuration is still supported in the new Files On-Demand experience if an external drive is selected during the first-run experience. A few things change as a result:
- The sync root remains in ~/Library/CloudStorage, on the user’s home volume. As noted above, this path cannot be moved from this location.
- The cache path is on the volume that was selected during the first-run experience. This is located in a hidden folder that’s a sibling of the location that was chosen.
- This folder begins with the name “.ODContainer”.
- A symlink is created from the chosen location to ~/Library/CloudStorage.
For example, if the user selects /Volumes/BigDrive/OneDrive for their OneDrive path:
- The sync root will remain in ~/Library/CloudStorage/OneDrive-Personal
- The cache path will be set up at /Volumes/BigDrive/.ODContainer-OneDrive
- A symlink will link from /Volumes/BigDrive/OneDrive to ~/Library/CloudStorage/OneDrive-Personal
Because the cache path is located on an external drive in this scenario, any pinned content will be stored there and not on the main drive.
The cache path folder is hidden by default. Users should not modify this folder or its contents.
User consent
For OneDrive to complete setup with the new File Provider platform, the user must consent to allow OneDrive to sync. This experience is like the experience of allowing an application access to the Documents folder or the user’s Contacts.
Consent is not required in the following cases:
- If the user previously opted-in to use the Finder Sync extension. This is set by default for the Standalone build, and the vast majority of our App Store users have opted-in as well.
- If the OneDrive app was deployed and managed through an MDM tool. MDM-managed applications are considered to have implied consent by the administrator.
If consent is required, the user will be prompted to provide it during the first run experience when setting up OneDrive for the first time.
The user can withdraw consent from the System Preferences -> Extensions preference pane. If consent is withdrawn, OneDrive will display an error dialog, an error in the Activity Center, and an error icon, until the user provides consent again. OneDrive cannot run without consent.
Always Keep on This Device
A standard feature of Files On-Demand on all our platforms is the ability to mark files as “Always Keep on This Device.” Internally, we call this operation “pinning.”
When a file is pinned, it is downloaded to disk and is always available offline, even if there is no network connection. The presence of the check mark icon indicates that a file is in this state. Folders can also be pinned, which means that all files and folders underneath the folder will inherit the state, and new files added to that folder will also inherit the state.
Pinning a file on the new Files On-Demand platform means that its contents will be downloaded into the OneDrive cache. Because is the file is in the OneDrive cache, it can always be provided to the sync root whenever it is needed, even if the machine is offline or the OneDrive app isn’t running. The presence of the gray check mark indicates a file that is in this state.
You may notice that pinned files sometimes have an icon next to them that indicates they aren’t downloaded. This icon just means that the file isn’t in the sync root. If a file has the gray check icon, it is still always available because OneDrive has the file in its cache and can always provide it.
Free Up Space
When you no longer need a file on your Mac, you can use the “Free Up Space” option to immediately evict its data. When you do this, data is evicted from both the sync root and OneDrive’s cache, ensuring it occupies no bytes on disk. However, it is still available in the cloud.
Disk space usage
Files that are kept in the sync root do not count against disk space usage, unless they are marked as “Always Keep on This Device.”
For example, imagine the sync root contains five files, each 20mb in size, for a total of 100mb. These files are fully in-sync with the cloud. Now imagine another application asks about the amount of free space on the drive. These five files do not count against the disk space used, so the size reported to the application will be 100mb larger than you might expect.
The reason for this behavior is that in low disk space situations, these files can be automatically evicted from the disk to make room for more data. For instance, imagine an application wants to write 50mb of data to disk, but there is no more disk space. However, because the five files in the sync root can be evicted as needed, the write can safely complete. To do that, three of these five files will be evicted to make 60mb of space, and so the 50mb write completes.
This behavior has several implications:
- Files that have data in the sync root can be evicted at any time.
- The system will automatically clean up files as disk space runs low.
- Mark files as “Always Available on This Device” if you do not want them to be evicted.
We know that some organizations have scripts or something similar to automatically free up space for OneDrive content, usually at login or on a set schedule. Because macOS will automatically free space from OneDrive files as needed, such scripts are no longer necessary, unless you want to prevent users from keeping content in the “Always Keep on This Device” state.
File system feature support
The new Files On-Demand experience supports some existing features of APFS that were previously poorly supported by OneDrive. These include:
- File tags
- Last used date
- File system flags
- Extended attributes
- Type and creator code
- Symlinks
Note that changes to these properties do not sync to or from the cloud, but OneDrive will preserve them on the local file system. Previously, they might only have been preserved for a short while but overwritten by a change from the cloud.
Symlinks have special support in the new experience. They are preserved as a symlink in the sync root but do not sync to the cloud as a symlink, as the OneDrive cloud does not support symlinks. Instead, the symlink will sync to the cloud as a plain text file with the symlink target as its contents. Previously, OneDrive ignored symlinks.
Packages
OneDrive now supports syncing packages, or files that appear as a single file but are actually a directory with many files and folders underneath them. Some applications exclusively create packages. Additionally, most Mac applications are stored on disk as a package.
Traditionally, the problem with syncing packages has been that packages often contain file states that don’t sync well in the cloud. For example, some packages contain internal symlinks, extended attributes, or other file system quirks that can result in a corrupt package if these are not synced correctly. The OneDrive app itself is an example of such a package – previously, if you saved the OneDrive app in OneDrive and attempted to open it on another Mac, it would be corrupt.
With the new experience, packages are now synced as a single file with a hidden .pkgf extension appended automatically. For instance, if you create a file named “Foo.app” in your OneDrive, it will sync to the cloud as “Foo.app.pkgf”. OneDrive automatically strips the .pkgf extension on compatible Macs, and the file will appear as a valid package on all compatible Macs.
Note that Macs not running the new Files On-Demand experience cannot read files in the .pkgf format.
Unlink, unmount, and reset
When you unlink your Mac or unmount a syncing location, OneDrive will preserve the non-dataless contents of your sync root. This works by removing the symlink to ~/Library/CloudStorage, creating a new folder in its place, and moving the files in your sync root that are not dataless to that location. Files in the OneDrive cache path are removed.
OneDrive also ships with a reset script included in the application bundle. This script behaves in a similar way, except that the non-dataless files are always moved into a folder in the user’s home directory named something like “OneDrive (Archive).” Files in the cache path are removed, except if the cache path is located on a volume other than the home volume.
Over time, we expect to improve this experience.
Learning more
To learn more about OneDrive,
- You can stay up-to-date on all things via the OneDrive Blog and the OneDrive release notes.
- Check out the new and updated OneDrive documentation.
- Take advantage of end-user training resources on our Office support center.
Thanks for reading!
Jack Nichols
Principal Software Architect - OneDrive
Posted at https://sl.advdat.com/3zVzEFV
