batsignal (no CVE) - a macOS LPE

Posted on 2023-10-30 in blog

UPDATE:

A couple hours after publication the Apple Security Changelogs were updated across the board, and they added me to CVE-2022-26704. I knew this was in the works, but it's still good to see. Thank you :)

This post is a writeup of batsignal, a macOS local privilege escalation bug from my talk:

Unexpected, Unreasonable, Unfixable: Filesystem Attacks on macOS

Background

This is a blogpost I'm dreading to write. It's about my first ever report in the ASB (Apple Security Bounty) and it was by far the worst experience I ever had in it. It took 332 days to get a bounty, after 2 bypasses and at least 1 collisions. There was a silent fix. There was me forgetting to send an email with a working exploit. It was the opposite of fun.

From report to bounty:

332 days passed
I sent in 2 exploits
- 7 total PoCs, if we count minor revisions
I exchanged ~40 emails with Apple

About the bug

batsignal is an unprivileged user to root LPE in Spotlight

The user can be any user on the system, including Guest.

I found it in November 2021 and sent the report to Apple a month later.

Due to collisions and fixes I have developed two variants: v1 and v2. There's v3 (alfred), but that is tracked as a separate bug, even though the underlying issue is the same.

Spotlight background

In case you are not aware of what Spotlight is, it's the search service on macOS. Since it indexes the disks on the system, it's an interesting service for security research purposes.

The Spotlight service consists of a couple daemons, the two that we are interested in right now are:

mds - runs as root and has FDA
mds_stores - runs as root

There is a third one: mdsync, but that's not our topic right now. Going forward, by Spotlight I will collectively refer to mds and mds_stores. They are doing slightly different things and have slightly different sandboxes, but both run as root and that's what matters.

To do it's job, Spotlight has to have access to files on the system and this is perhaps the reason why these daemons run as root, I'm not entirely sure.

We won't investigate the file parsing aspect of the service right now, but instead we will take a look at a curious behaviour I noticed while poking around: the index files of a volume are stored on the volume. Since this works with disk images I can provide, it opens up some interesting avenues to exploitation.

It's worth noting that this is a classic speed vs security tradeoff: On the one hand, carrying the indexes with the volume keeps it available and keeps searches fast. This is good. On the other hand, an attacker can use this feature to try and get remote code execution on a machine or to escalate privileges locally. This is bad.

I'm only bringing this up because Spotlight does have the ability to store these index files locally: it does so for network volumes, but not for local disks. Obviously this was a conscious decision and if I had to speculate I'd say it's done so that volume indexes don't fill up the valuable local disk. The takeaway here is that this is not likely to change.

The bug

Spotlight does file operations on the volume in a SIP-protected directory called /.Spotlight-V100.

I don't think there is a point to talk about a concrete bug here, as this is a design issue. The above behaviour is the bug.

At the risk of confusing you, we have to take a detour. You see, running a root daemon on a mounted filesystem by itself is not a huge deal. This is true with one major caveat: It's only safe to do so if you can guarantee that the volume is trusted.

So, on a normal Linux system - as far as I know - a regular user can't mount a volume. Furthermore they can't trigger a root daemon to operate on the freshly mounted filesystem. I might be wrong here, so send a DM if I am.

This is different on macOS. On macOS - to my horror - disk mounting is so ubiquitous that it's one of the first things any user does after installing the system. No prompts, no passwords, no problems. As it happens, also no security.

Before you think that I am off my meds, spewing hyperbole on hyperbole: let me introduce you to some nastiness that is allowed for all users on macOS, including non-admin ones.

Any macOS user can:

mount their own disk image
umount, update or move the mountpoint
set mount flags like union and noowners

Here are a few ways this can help an attacker:

For #1 they can prepare a malicious image.

For #2 they can "rugpull" daemons, change mount flags, etc...

For #3 they can simply use noowners so that they can write to any file that only root should be writing. Arguably the same could be achieved using other means - like inheriting ACLs - but this is more convenient.

With all of these amazing tools we can easily disarm SIP, since as far as filesystems are concerned SIP is just a regex engine.

So, how would we break a regex engine?

Universal SIP bypass on user-mounted volumes

To protect the .Spotlight-V100 directory, SIP will try to match it by name. It - most likely - uses the directive mount-relative-regex. I'm saying most likely because that's how it allows Spotlight daemons access to it. This is available in the sandbox policies of mds and mds_stores, but the mechanism by which it prevents everyone else access is hidden from us. Thus I can't say with 100% confidence that this is the same mechanism, but it's certainly a safe bet.

With these assumptions in place, we already know how to break the protection. We need to change the name. Obviously this is not allowed while the volume is mounted, but we can defeat SIP by unmounting it. Editing the disk image is like editing anything else: Now it's not a filesystem anymore, it's just a precise collection of bytes.

There are two major ways to attack: offline and online:

offline means we can prepare the disk in advance but we can't mess with it while it's mounted
online means that we can manipulate it while the disk is mounted and the target application is running

While I had some partial success using the online method, it was usually too limited, with the exception for the simplest exploit, v1. Here it was enough, and was used for mostly convenience reasons.

Developing a more generic and powerful online attack is still on my todo list. With that said, I will stick with the offline method from now on. It's more limited, but good enough for now. It also has the added benefit of not requiring command execution on the host, making it an ideal infection vector.

For the offline (unmounted) disk manipulations we can mess with any protected file/directory:

we can delete / modify / replace
we can hardlink to the volume's root for later access
we can modify permissions
etc...

Before we manipulate the image, we have to pick our filesystem. There are many choices, but in my tests HFS+ was the best. You might have luck with APFS, HFS (the legacy one) or various HFS+ modes like, journaled, case sensitive, etc... You might want to mess with VFAT (MS-DOS on macOS), or anything else as long as Spotlight uses it as a backing volume for the indexes. Sadly, network filesystems won't work. That'd be too easy.

For the curious ones here is the full list of filesystems that you can mess with using the command line right now:

mount_9p        mount_afp       mount_cd9660    mount_devfs     mount_fdesc
mount_hfs       mount_msdos     mount_smbfs     mount_udf       mount_webdav
mount_acfs      mount_apfs      mount_cddafs    mount_exfat     mount_ftp
mount_lifs      mount_nfs       mount_tmpfs     mount_virtiofs

This list is not exhaustive by the way, these are just the ones that come with a console mount command. There are others filesystems supported by xnu proper as well:

$ ls xnu/bsd/miscfs/
bindfs  deadfs  devfs  fifofs  Makefile  mockfs  nullfs  routefs  specfs  union

Also there's 3rd party ones, etc... I don't want to digress too much. I will be using HFS+ for the rest of this, you'll see why a little later.

Now we have the filesystem taken care of, what now? Well, we want to make it do things it really should not do. I found two good ways to make this happen.

#1 using a real filesystem driver:

We could use Linux as it has support for a lot of filesystems, and it understands HFS+ just fine. It has much less restrictions than the macOS HFS+ driver, so we can use it to create forbidden structures. You have to use force parameters to get the image to mount as rw, but for my small scale tests everything worked fine.

Alternatively, we could use anything else that can reliably write to the filesystem of our choice. For this FUSE is an obvious choice, but I did not experiment with it. Nevertheless, it should work fine.

#2 editing the binary directly:

A more barbaric - yet effective - solution is to just edit the binary by hand. During one of my quick and dirty tests I noticed that I can mess with the HFS+ volume by search and replace. This worked shockingly well as long as I was somewhat careful.

It's worthy to note that this is not really possible for any complex - typically newer - filesystems, only simple - typically older - ones like FAT and HFS+.

Since this was reliable and quick to do locally, I used it in all of my exploits. I could get away with this as I only needed to change a string on the filesystem. For anything more complex you really need to use a filesystem driver.

So, with HFS+ we can simply do the following:

buf.replace(b’\x31\x00\x30\x00\x30\x00’, b'\x39\x00\x30\x00\x30\x00’)

The Python code above would change 100 to 900, resulting in .Spotlight-V100 -> .Spotlight-V900. This change is arbitrary, any other regex-breaking modification would work. You could also use a larger signature if you don't exactly match in 2 places. If the signature matches 2 locations, you are good to go.

With this dumb tool in our arsenal we can already see some good results.

So, to bypass the SIP protection:

we create and mount an HFS+ disk
we turn Spotlight on with mdutil -i on /MY/VOLUME, so /.Spotlight-V100 gets created on the volume
we unmount
we edit the binary image to break the regex match
- turning .Spotlight-V100 into .Spotlight-V900 (or anything else)
we mount
we do whatever we want under .Spotlight-V900 :)
we unmount
we reverse the binary edits
we mount again
- this time Spotlight will run and operate on malicious content
- optionally we can trigger a Spotlight index rebuild with mdutil -E /MY/VOLUME

Now this is all well and good, but so far we did nothing useful. Let's fix that.

batsignal v1 - symlink attack

Architectural mistakes aside, one of the concrete bugs in Spotlight was the naive way it wrote files. Since Spotlight (correctly) assumes that it's protected by SIP, it doesn't really make an effort to work with files carefully.

SIDENOTE:

I've often seen this in "defense in depth" systems. Once components realize there's a "mitigation" protecting them, they won't focus development effort on security. This immediately defeats the attempt to have "defense in depth".

Knowing this, we can invert our general thinking and think about systems like this not as "hardened" but as "low-effort-enabled".

In Spotlight we can trigger insecure file writes by writing a "cacheable" file on the volume. When this happens, the file's content will get indexed and the extracted text gets written to a new file in the Cache directory.

This would result in a file called ./mnt/.Spotlight-V100/Store-V2/[UUID]/Cache/0000/0000/0000/[X].txt Here [UUID] is a UUID, and [X] is the inode number of the original file.

Many file formats are supported, but for simplicity I went with a pdf called payload.pdf, the content of which was one line of text: ALL ALL=(ALL) NOPASSWD:ALL.

There is one minor twist. First [X].tmp would get written and it would get rename()d to [X].txt. The write would be done with open() / write(), and in the open() call symbolic links would be followed.

This is clearly a major vulnerability.

While trying to exploit this however, it became apparent that yet another issue exists:

SIP prevents Spotlight from actually following symbolic links that point outside the volume. This was a smart move on Apple's part.

Luckily for us, the policy files explicitly enable Spotlight to write to /Library/Caches/com.apple.Spotlight. This also includes the permission to follow links to this location. Since that directory is writable by our user, we can just hardlink /etc/sudoers there as [X].tmp. This defeats the extra protection.

NOTE: In my slides and presentation I omitted this step for simplicity, which might have been confusing. Sorry about that.

To exploit:

create a disk
mount (at ./mnt)
copy payload.pdf to the volume
turn Spotlight on
wait for the payload file to get indexed
- this step creates the Cache directory and files
unmount
do the disk image edit trick
mount
move the Cache folder to the root of the volume, and replace it with a symlink
- NOTE: this ^^^ step is optional, but really convenient for debugging as we can edit the contents online
- this symlink will be followed, since it's within the volume
unmount
reverse the disk image edit
mount
create hardlink to /etc/sudoers as /Library/Caches/com.apple.Spotlight/whatever
create a symlink to /Library/Caches/com.apple.Spotlight/whatever at ./mnt/Cache/0000/0000/0000/[X].tmp
- we can do this since ./mnt/Cache is the real cache directory
delete [X].txt from Cache
issue a reindex
at this point /etc/sudoers would be overwritten with our payload

That's all there is to it.

Roughly two months after sending this in I noticed a fix in the current beta that killed this vector:

Spotlight was disallowed from accessing /Library/Caches/com.apple.Spotlight/. Bummer.

batsignal v2 - union mount attack

Thinking a bit about bypasses I managed to figure it out fairly quickly: A week after the fix in the beta I have sent in information about the bypass. Crucially I didn't include a PoC. This was a mistake on my part.

The bypass was simple:

Just use union mounts, since SIP's mount-relative-regex is completely blind to them, or rather union mounts are too transparent.

To exploit (this is mostly from my original report):

create a volume
mount it, copy payload, activate spotlight
wait until Spotlight creates the Cache folder and contents
get the UUID Spotlight generates from /Store-V2/[UUID]/Cache/
umount
create the same directory layout outside the volume: ./mnt/.Spotlight-V100/Store-V2/[UUID]/Cache/0000/0000/0000/
remount with union
issue a reindex - This time spotlight will erroneously use our directory outside the volume, allowing us to use hardlinks
hardlink the [X].tmp (18.tmp on my machine) file to sudoers as before
reindex to trigger the overwrite of 18.tmp with our content

This is not terribly different from the previous one, but it achieves the same end result. Since I didn't include a working PoC at first, Apple never got back to me.

I requested an update about a week later at the end of the original 90 day deadline. They said they are "investigating", and asked me if I am willing to withhold disclosure. I agreed. A few days later they requested more information and thanked me for my cooperation.

I sent a working PoC the next day, the first iteration of batsignal v2.

There were 4 minor revisions of v2. One of these (the 3rd one) I just simply completely forgot to send. That was my mistake, I only realized that after roughly a month of silence from Apple. Oops.

About 3 months later I sent the final revision of v2. I won't bore you with the details of all of these versions, they essentially did the same thing.

Yet another 4 months - and a lot of back and forth - later I received the email about the bounty.

If you like timelines, there's a detailed one at the bottom.

The fix was simple:

Spotlight can not access files on union mounts anymore.

I can already spoil the ending for you: it's not going to be enough.

Demo and code

Demo video

Here's the exploit code: https://github.com/gergelykalman/no-CVE-batsignal-a-macOS-LPE-in-Spotlight

Conclusion of the bug

In conclusion there are major design issues that exist on macOS that are unlikely to get fixed. I demonstrated in the post how attempts at patching this failed repeatedly. The newest bypass, alfred (which is essentially v3) will get it's own writeup soon, but the story is not over yet.

I suspect there will be many more bypasses to come, for one simple reason: an attacker has too much leverage.

To sum this up succintly:

Securing file accesses of privileged applications is impossible if an attacker can control the filesystem.

The bounty

For all of my efforts of sending bypasses, ~40 emails, countless retests and waiting for 11 months, I was finally awarded with a bounty of $17,000.

I was conflicted whether I should talk about this at all, but I feel like there isn't a lot of good information out there about concrete bounties. Apple certainly is not helping this transparency and I understand why. Bugs are really hard to price.

With that said, the bounty amounts on the security website are misleading to a degree that - at the time - I felt like I was cheated. I still feel conflicted about it, even though I have sent in many other reports and I ended up having a much easier time with those. For the newcomers to the program, and for the unfortunate ones who do experience collisions, I felt like it was important to write this down publicly. Also this is somewhat therapeutic for me personally, as I had to spend close to a year with my mouth shut.

I hope this part provides some value to the people who are frustrated by Apple's bounty program, and if you are not interested in my self-indulgent ranting, please feel free to skip it.

Did I get ripped off?

The amount was a lot less than I expected and I was thoroughly bummed out. After feeling angry and somewhat sad, I talked to some friends which helped a lot. Some of them suggested that I should rethink this. So, to do that I needed to figure out a way to come up with an objective valuation.

This is really hard to do without personally being invested in the bug, but it's especially difficult if you are. Nevertherless, I tried to set aside my own biases and expectations.

The Apple security website did not exist then, but it was known that a macOS LPE is worth more. The - since released - security website says this is a $50,000 bug on iOS, starting from the sandbox, with a quality report and exploit. Please note that this is the maximum payout.

To get a somewhat objective valuation, let's see what I have.

Since app sandbox escapes generally go only for 5k, that requirement can pretty much be ignored, since - correct me if I'm wrong here - it doesn't matter whether parts of a chain are submitted together or separately. Thus I'm happy losing 5k and not have to deal with an app sbx escape.

As for the report quality: mine was definitely not the best, but it was not bad. I sent in multiple valid exploits, performed extensive testing of betas and even sent in bypasses. I would not expect to get a severely reduced bounty here.

Now for the big one: My bug is on macOS not iOS, so it's definitely worth less. It has been publicized that Apple had about 2 billion active devices in January of 2022. Looking at the latest earnings report it seems that iOS outsells macOS pretty consistently by a factor of 5x-7x (source). iPads and other wearables account for about another macOS sized chunk, but the big deal here is obviously iOS. I know that this is only the last 9 months though, so I looked at statcounter.com for futher data. According to them, iOS is about 2x as large as macOS as far as active devices are considered.

This whole analysis might be totally off the rails so I don't want to digress further, but these numbers - roughly speaking - make sense to me.

Taking all of this into account, does 17k still seem like a reasonable bounty? It kinda does. Since it's roughly 1/3 of the 50k - which is the maximum anyway - it is not something I can stay too mad at for too long. It's acceptable if we can take the maximum payouts at face value.

This also brings us to the bigger question:

Do we think it's okay for one of the richest companies on Earth to award researchers with a maximum of $50,000 for responsibly disclosed, weapons-grade exploits + documentation? Exploits with a blast radius of more than a billion devices, - and not only that - devices that are specifically marketed as being secure?

This question is silly and rhetorical: Apple pays as much as Apple chooses to pay.

There's not a single thing me or you or anyone else can do to change it, so the real question for us is whether this is worth it, for us, researchers.

I will take a deep dive into that mess in a future blogpost, but as far as this one is considered, I'm glad it's over. I hope you are too.

Timeline

2021.12.12: Report sent to Apple
2022.01.11: Issue was reproduced
2022.01.18: Fix promised in Spring 2022
2022.02.25: I noticed there's a fix in the current beta that removes a crucial directory from the SIP profile, breaking the original exploit. I told Apple this and that I think I can bypass the fix.
2022.03.03: First bypass sent to Apple
2022.03.05: Apple: investigating the potential bypass
2022.03.14: Request for update
2022.03.16: Apple: still investigating, will you still withhold disclosure?
2022.03.22: Apple: we need clarification (fair enough, I didn't send a PoC, just a writeup)
2022.03.23: I sent a PoC (the first version of batsignal v2)
2022.03.25: Apple: reviewing
2022.04.04: I ask for an update, tell Apple that the bypass works on 12.3.1 still
2022.04.12: Apple: still investigating
2022.04.19: I ask for an update tell Apple that the bypass works on 12.4 (21F5058e) still
2022.04.26: Apple: "We are tracking this as a non-near term solution."
2022.05.16: Released fix for CVE-2022-26704 (Description: A validation issue in the handling of symlinks was addressed with improved validation of symlinks.)
2022.05.22: I notice CVE-2022-26704 in the changelog, ask for an update as it was originally credited to anonymous
2022.05.27: Apple says this is not as a result of my report
2022.06.30: I sent in the final version of batsignal v2
2022.07.06: Apple thanks me for the new PoC and says that NO change has been made due to my report
2022.07.20: Released fix for CVE-2022-32801 (Description: This issue was addressed with improved checks.)
2022.07.28: I saw the fix in Ventura so I told Apple, also told them that this is bypassable
2022.07.29: Apple appreciates the update and says I should open a new ticket if I can bypass the fix
2022.08.22: I notified Apple in the old thread about the exploit working perfectly on latest 12.5.1 still
2022.08.23: Apple: no changes have been made due to your report in 12.5.1
2022.09.13: I noticed a silent fix in release 12.6, asked Apple about it
2022.09.16: Apple says NO changes were made as a result of my report in 12.6, but they're tracking changes Ventura, asked how I'd like to be credited
2022.09.16: I realize that I collided with Joshua Mason, told Apple
2022.10.26: Apple confirms additional changes required due to my report, asks for the postponing of my disclosure. They say I should get an update regarding my bounty in a few weeks
2022.10.26: I confirm that I will wait for the fix
2022.11.09: Bounty awarded :)

Special thanks

Thanks to Csaba and Wojciech