A
Story
Whose code am I running in GitHub Actions?
  • dgl
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
Unfortunately this makes a mistake by using a short commit ID: "(e.g. a5b3abf)"

That's not a full commit ID, so it can still result in a mutable reference if either someone can find a clash[1] or if they can push a tag with that name and it takes priority in the context it is used (this is somewhat complex, e.g. GitHub prohibits pushes of branches and tags which are exactly 40 hex characters long, but other services may not).

[1]: https://people.kernel.org/kees/colliding-with-the-sha-prefix...

Shortened commit SHAs are actually not supported by Actions; if you try, you get

"Unable to resolve action `actions/checkout@11bd719`, the provided ref `11bd719` is the shortened version of a commit SHA, which is not supported. Please use the full commit SHA `11bd71901bbe5b1630ceea73d27597364c9af683` instead."

What if the repository has a tag called 11bd719? Does Git/GitHub forbid creation of this tag if a commit exists with that prefix?

What if a Git commit is created that matches an existing tag? Does Git have a procedure to make a new one? e.g. imagine I pregenerate a few million 8 character tags and wait for a collision

btw: Even if you specify the full commit SHA, this can still be attacked; there have been pre-image attacks against Git commit hashes in the past. At least for older versions of Git, the algorithm was Sha1. Maybe that’s changed but an attacker could always construct a malicious repository with intentionally weak hashes with the intent of later swapping one of them. (But at that point they may as well just push the malicious code in the first place.)

What is the attack exactly? Only full commit SHAs are valid to reference a commit by SHA. GitHub disallows tags and branch names that could collide with a full commit SHA. There is never any collision between commit SHAs and tags.
I think the hypothetical attack is to create a tag with the shortened commit SHA pointing at malicious code, and if someone accidentally puts that instead of the full commit SHA, maybe Github will serve them that malicious tag instead of throwing the error. It sounds like that could work if Github doesn't block a tag/branch colliding with a shortened commit SHA. I'd guess they probably do though?
So you would need to specifically write an action referencing an invalid short SHA, which would not work and the action would fail, and then wait for an attacker to push an action with that tag name, and then run your action which has thus far been failing because of the invalid reference?
You'd push the tag at the same time you push the commit. If anyone tries to reference your action and accidentally copies the shortened commit SHA instead of the full commit SHA, they'll reference the malicious tag instead. They'd never see it fail, they'd just silently pick up the malicious tag. But again I'm guessing Github will block that shortened commit SHA as a tag and this wouldn't actually work.
No, I don't think Github blocks shortened commit SHAs as tags.
How could they? They can't block every 8 character tag. And you can push the tag before you push the commit. (You know which short sha to impersonate because you can see it locally.)
> They can't block every 8 character tag.

Whilst Git will be default abbreviate commits to 7 characters, that's merely a default; `core.abbrev` can be set to any number to change the default display. Git will also accept any length abbreviated hashes as long as they're unique in the repo.

  • ·
  • 3 weeks ago
  • ·
  • [ - ]
It's still SHA-1 by the way, but they included counter-cryptanalysis to reject objects that appear to be one side of a collision using known techniques.
So just so I'm clear based on what you've mentioned, even the policy prohibiting 40 hex character tags isn't doing anything to stop a tage the same as the short commit ID?

Also, per this comment on a previous discussion on this incident at https://news.ycombinator.com/item?id=43367987#43369710:

> the real renovate bot immediately took the exfiltration commit from the fake renovate bot and started auto-merging it (updating full SHA1 references)

SHA pinning won't necessarily help if the dependency you are pinning doesn't pin its own dependencies! You still get stuff pulled via vulnerable tags etc. How long till we get this https://github.com/github/roadmap/issues/592 ...
Yes, this is a crucial distinction to make. The fact of the matter is that you have to treat GitHub Actions like a compromised system. Sure, there's not a ton of steps you can take for protecting builds if it's your primary builder, but you can for example not hook up an AWS account with full admin privileges to it (which I've seen more times than I would have like to).
https://github.com/features/preview/immutable-actions

They are actually releasing this very soon. I’ve seen some of my workflows use an immutable OCI image for some of GH’s actions like actions/checkout.

Isn't that wrong? I think you have to pre-bundle your actions, it won't do an npm install.
I set up this recently at a new company and did yarn + ncc to build a compiled js out of typescript. It was a bit hairy as a novice, but ended up working fine.

That protects from npm supply chain stuff, but obviously third-party includes like docker/build-push-action are still a risk.

Thanks for highlighting this open issue.

The fact they've been stalling this for a good 2.5 years is... insane??

I don't believe that's true. If you pin to a hash, then it will always run that version and can't change
  • ptx
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
There seems to be a slight misunderstanding in the article. It says that the "v2" tag "looks like an immutable reference" and points out that it's actually mutable, as if this was surprising and unintended. It also says that the reason people use tags despite this (making a tradeoff against security) is that "tags are easier to read and compare".

But the GitHub documentation [0] makes it clear that tags for major versions are intended to be mutable and be updated to point to new minor versions as they are released, not because it's "easier to read" but because you "can expect an action's patch version to include necessary critical fixes and security patches, while still remaining compatible with their existing workflows" (as long as the author follows their recommended semantic versioning scheme).

So choosing a major-version tag is GitHub's recommended practice precisely because it is mutable and does change.

[0] https://docs.github.com/en/actions/sharing-automations/creat...

> major versions are intended to be mutable and be updated to point to new minor versions as they are released [...] because you "can expect an action's patch version to include necessary critical fixes and security patches [...]

It's two sides of the same coin: on one hand, an update can include fixes for bugs and vulnerabilities; on the other hand, an update can also include new bugs and vulnerabilities (or even malicious code). Updating too quickly can be risky. Updating too slowly can also be risky.

While this may be technically correct, the general git-community at large mostly treats tags as immutable (contrary to docker, for example).

Release branches are typically the mutable reference. So I would create a 'v2' release branch, but not a 'v2' tag which gets updated.

Also, by convention, git references starting with 'v' are typically immutable tags and not branches.

But, even given the above, the git-community at large knows that tags can be mutable, and so if we care about that, we reference the sha (malicious collisions excepted).

  • ptx
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
That's a good point and might explain the source of this confusion. Actions on GitHub can come from either Docker or a Git repo, using exactly the same syntax [0], so the tag can be either a Docker tag or a Git tag.

[0] https://docs.github.com/en/actions/writing-workflows/workflo...

  • bsza
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
That wording is even more misleading, because it implies that using the full version string, by contrast, is not mutable, even though it presumably is.
I just started using GitHub Actions for a personal project, and as you do, I trawled HN for opinions on how to use it.

At first I built a workflow out of steps published on GitHub. Use ilammy/mms-dev-cmd, lukka/get-cmake, lukka/run-vcpkg, all to build a project with CMake for Windows targets. Of course I referred to actions by SHA like you should

   uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756
But one comment stuck with me. Something like, “You should just run your own code on GitHub Actions, rather than piecing it together from publicly available actions.” That made a lot of sense. I ended up writing a driver program for my personal project’s CI builds. One job builds the driver program, and then the next job runs the driver program to do the entire build.

I wouldn’t do this if I were getting paid for it… it’s more time-consuming. But it means that I am only minimally tied to GitHub actions. I can run the build driver from my own computer easily enough.

> I ended up writing a driver program for my personal project’s CI builds. One job builds the driver program, and then the next job runs the driver program to do the entire build.

Yes things like that have been discussed before on HN. Also for example use a justfile (or something similar) and then call that from inside the Action to reduce vendor lock-in.

I use Actions merely as a way to trigger a custom Webhook. Then I do everything on the server that receives the hook with my own code. I hate YAML that much.
What I want is something like “please run this command on a server somewhere when event X happens”. Seems like the options are along the lines of:

1. SaaS CI/CD products, like GitHub Actions,

2. Run your own Jenkins cluster,

3. Figure out how to orchestrate cloud resources to do this for you.

Maybe there are easy options that I’m missing. I don’t really want to create docker containers just to build some program I’m working on.

That's effectively what we are doing. The webhook receives any "custom properties" you have defined on your repo, the ssh url, and critically, the name of the Action that was run. The receiving server can use all of this to select the appropriate pipeline. Our build server is not containerized.
Listen to a port from a server. Do a post with an API key. Then run your bash script there.

Run GitHub actions self hosted (takes 2 mins to setup)

Just ssh in and run it.

So many options.

Sounds like you are assuming that I have a server always running for this stuff? That assumption is wrong. I don’t want to run CI servers. If I had servers always running, I would install Jenkins on them and the problem would be solved.
I don't understand your issue.

If you are doing deployments, actions etc does exactly that. Run pure bash commands or whatever.

If you want it for other purposes, you essentially want to run a "server" application but don't want to manage a server. Just use serverless? Write a JS function (or some other languages) and the platform will run it when the event triggers.

  • arccy
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
why not just use regular GitHub webhooks...
We do, but you can only trigger those on predefined events, and we want our release manager to be independent of any push or pull mechanisms on the repo. You can also run actions from the github web ui which makes them available even to non technical managers.

Our Action has a single step, it has an "if: false" declaration so it never runs, and no runners are engaged. This immediately completes and fires off a "workflow_job" webhook which triggers the build server to act.

  • djoel
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
Do you mind linking the repo, if it's public? Thanks!
Github Actions is definitely a vector for abuse.

I was looking at Seleniumbase recently, and they tell you that you can use Github Actions for web scraping to bypass a lot of blocks (apparently Github Actions use a residential IP-space)

https://seleniumbase.com/new-video-unlimited-free-web-scrapi...

This seems like a wild thing for a third-party project to promote. The intention of GitHub Actions to run CI/CD and other repository-related tasks. You’d never see, for instance, Adobe promoting, via YouTube, “unlimited free web OCR with Adobe CLI on GitHub Actions!”

I’ve never heard of Seleniumbase, but this makes them look like a rinky-dink project.

That's the whole point of hacking is to use something in a way unintended by its maker. That could be for something cool/interesting, or it could be for something nefarious. Nobody ever thought a coffee maker should run Doom, but they do. Not sure if there's a morality clause type of dis-qualifier for a Show HN, but there's a lot of people that would be interested in seeing how something benign was used for a different purpose. Especially if if saved them money/compute/time/resources/etc.
Yep. And "hackers" who apply that mindset to abusing publicly shared resources are why the rest of us are going to get DRM on our own, private coffee makers.
You skipped the step where the manufacturers make the physical item in your home a SaaS requiring wifi access and an account in order to enable all of the features advertised on the box.

Your comment makes it like all attempts at hacking are nefarious. Some are in direct response to the makers being assholes and attempting to extort more money from the same sale. You want to lump all hackers into the same box, yet I want to lump all manufacturers into the same evilCorp box.

My comment is that this specific case of "hacking", that you directly replied to, is a destructive expression of predatory greed.

You're the one lumping them, as well as my comment, into false dichotomies.

I've found this to be a problem that a lot CI providers suffer from. They allow extension via third party code which is awesome, people write useful code, a lot of that useful code doesn't get maintained properly or ever, rots, and eventually everyone has a security issue.

You can also see the GitHub IP space here, I don't think it's "residential", unless that terminology includes azure and aws?: https://api.github.com/meta

I'm not sure. Perhaps when you call a browser it's going through another network? I haven't tested this for myself, only going off of of what was reported by that project.
Maybe it was a self hosted runner? I run those locally all the time.
  • bri3d
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
They don't use a residential IP space; they use Azure Data Center (which, being less popular, isn't blocked as often as for example EC2).
Network enabled compute is definitely an unusual free lunch, but I suppose the trade off is handing out free source code.
This does not bode well for genetic AI
  • dqh
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
I've never liked the idea of community actions in the critical build path, so I use official actions/* when I can, and otherwise use actions/github-script to invoke the GitHub API via inline JavaScript when I can't.
  • Uvix
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
I agree on community ones, but I’m happy to use the official actions from vendors like the Terraform and Azure ones too.
> At a glance, this looks like an immutable reference to an already-released “version 2” of this action, but actually this is a mutable Git tag. If somebody changes the v2 tag in the tj-actions/changed-files repo to point to a different commit, this action will run different code the next time it runs.

The worst part of this is that this is BY DESIGN.

I maintain a small handful of actions. You are expected to, as an action maintainer DELETE and RETAG your major versions when you release a new minor or patch version. That is to say for instance your v2 tag should point to the same commit as your latest 2.x.x tag.

Not everyone does this mind you, but this is the default and the expected way of operating.

I was frankly kind of taken aback when I learned this. I know for a fact documentation of this used to exist, but I am failing to find it currently.

You can see GitHub themselves however doing exactly this here, with the v4 and v4.2.2 tags matching here (as of today, v4 will move in future)

https://github.com/actions/checkout/tags

This is probably the dumbest design decision of GitHub Actions. You'd think that the biggest git platform would know better than asking you to force push every release. They should have just used branches, because that's exactly what they're for. Or they should have resolved the right tag themselves, like npm does.
I only use a handful of official github and docker actions. This behavior is something that I (and many others) want in that case.

Here @v4 is very similar to how you'd tag docker image v4-latest.

Ultimately, it should be a choice. Like you can do with package.json. Pin an exact, allow patch updates, allow minor updates or always latest etc.

Oh. I always assumed that v2 here was a branch, not a moving tag.
This article appears to be in response to the linked Tj-actions/changed-files GitHub Action Compromised – used by over 23K repos discussed at https://news.ycombinator.com/item?id=43367987 10 days ago; not a duplicate as it discusses a detection tool but perhaps it rhymes.
Was this an auto-generated comment? Because yes, of course it's related, it's someone doing their own investigations based on the news.
I tried to point out that though the topic is a duplicate, the link does add additional value. Do you not think the previous discussion is worth linking? Yeesh
  • ohgr
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
This is a minor worry when the entire software ecosystem is based on “download any old shit off the Internet at run it”.
My company went into full panic mode after this. 5 minutes later dependabot opens and auto merges a random patch from npm, but that's fine.
Say what you want about dependabot, and people who allow it to auto-merge changes, but at least NPM releases are not mutable (... anymore, at least. NPM had to learn that one the hard way, but unlike github it actually learned something).
> Tags vs commit IDs is a tradeoff between convenience and security.

Well, it's also a trade-off between security and security. If you specify an immutable commit ID, then if the dependency releases a security update you won't get it until you notice and update the commit ID. If you specify a tag, you'll get it on next build.

I guess we need Dependabot updating commitID's in workflows too? But then they'd update you to the vulnerable new @2, how would they know otherwise if it hadn't been reported yet?

Wait, people actually just blindly paste together calls into GitHub actions written by someone else who can change it at any time?

You know what, no this makes perfect sense. This is exactly, perfectly in line with the modern software ethos.

Jesus. I'm so glad that 100% of my GitLab pipelines is code I wrote. It's owned by the company and it lives in our source control and runs on our hardware. I think you'd be nuts to do anything else, honestly.

For entirely related reasons, I'm thrilled that my career is moving in a direction away from devops and software in general. I can't stomach it anymore.

Most people use several hundreds millions of lines of code provided by somebody else on a daily basis (your laptop, your phone, your hair dryer, car, etc). Most of that stuff gets built using libraries, components, frameworks, etc. provided by third parties.

The whole system runs on trust that all those people do the right things. Sometimes that trust is broken. But mostly it's surprisingly fine. Part of the reason is that bad people are the exception and not the norm and all those other people react when we find one, some are mildly paranoid about this, and processes exist for flagging suspicious things (e.g. CVEs).

What we need is not to audit everything ourselves. Because that's humanly impossible. But better trust verification mechanisms and tools. Github has some mechanisms for actions but it still has some vulnerabilities. It's not perfect. But it's better than nothing. Replacing those by auditing/building yourself is going to either result in a lot of work or security with holes in it (i.e. you are moving the problem, not solving it).

You could argue that most GH Actions are simple enough that building yourself is not the end of the world. It depends on what you are doing.

I take the middleground. I use GH actions but only with widely used actions maintained by Github. Actions are just docker containers. So, the advice can be generalized to those. Check where they come from; who is building them; what their release practices are. Etc.

> just blindly paste together calls into GitHub actions written by someone else who can change it at any time?

That’s old news. Now they have the AI do it.

Before this article I had no idea this was even something you could do let alone something that's common.

Why would you not just copy the couple of lines in to your own config. It's not like you need to subscribe to updates on a command to get changed files. You want the exact opposite so your CI doesn't randomly break due to external changes.

And then 2 years later you discover there was a security bug in that code you copied..
A security bug on a one liner regex to select some files in my own repo that changed? Seems far fetched.
Where to (just out of curiosity)?
  • ph3t
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
GitHub’s dependency graph is supposed to give us this kind of visibility without any custom scripting, but from my experience it’s pretty spotty and often misses dependencies entirely.

Also, the script from the article doesn’t cover transitive GitHub Actions dependencies. So if a third-party action you’re using relies on a vulnerable action internally, it won’t catch that.

  • ippem
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
https://github.com/orgs/github/projects/4247?pane=issue&item... - GitHub is working on immutable actions. Let's see then how they really work...
There is a breakdown in UX if your references need to look like this

SHA `11bd71901bbe5b1630ceea73d27597364c9af683`

That means absolutely nothing to a human being Whereas

> hashicorp/setup-terraform@v3

Is easy to understand.

How about

> hashicorp/setup-terraform!v3.0.1

This version, and only this version, fails if the version cannot be found.

Or

> hashicorp/setup-terraform@v3-2025-03-26-15-01

Give me the version of the library as it was at this specific time or fail if it can't be found

Presumably it would be wise to check the SHA as well to ensure no changes have taken place maliciously

One could go old school and import the code from the various needed libraries, into the main project. Now you have complete control over what runs and it can be audited and will be in a safe state until explicit action is taken to update the code.

  • junon
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
This is rehashing (no pun intended) a very long-discussed issue about versioning. Your post also contradicts itself.

> SHA `11bd71901bbe5b1630ceea73d27597364c9af683` - That means absolutely nothing to a human being

> Presumably it would be wise to check the SHA as well to ensure no changes have taken place maliciously

That's exactly why the first one happens so often - I've checked the dependency at that version, and I want to make sure I only get that version, as spoofing sha's in a Git context is not part of my threat model.

That is true. In the post I am replying to the idea was

SHA `11bd71901bbe5b1630ceea73d27597364c9af683

as what the script contains as a reference. While i would advocate

hashicorp/setup-terraform@v3-2025-03-26-15-01 Hash: `11bd71901bbe5b1630ceea73d27597364c9af683`

I consider it easier to deal with, .

  • duped
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
Actions are not immutable, period: https://github.com/github/roadmap/issues/592. This issue has been open for years and continues to be pushed back, so I would not recommend anyone use any github action as a dependency if they want to protect against supply chain attacks (basically, don't use GHA at all, it's a horrible product if you care about cost/performance/security).

You also cannot trust a git commit ID as an immutable reference. Even if you have a checksum available, you have to validate it.

  • Jenk
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
This has been in GH's docs on security hardening for a while[0], and I can't recall which tool it was, but I have seen reports that warn when not using SHAs. Pretty sure there was a linter that would even show the warning in my neovim setup that uses some kind of gh action LSP, but it has been a minute.

[0]: https://docs.github.com/en/actions/security-for-github-actio...

Testing for unpinned actions is supported in CodeQL, the security-extended suite
When I saw the tj-actions attack, I decided it was time to finally implement action wrapping with our `witness-run-action`. This will generate signed attestations on exactly what the actions are doing.

We have some more testing to do before we cut an official release, but it is working correctly for the limited cases we have tested it with. I'd love this group's feedback.

https://github.com/testifysec/witness-run-action/tree/v1.0.1...

> Tags vs commit IDs is a tradeoff between convenience and security. Specifying an exact commit ID means the code won’t change unexpectedly, but tags are easier to read and compare.

Imagine if we could specify both the tag and its commit, and the runner would check, at run-time, whether the specified tag is still pointing to the specified commit. This would essentially "lock" the dependency. Although storing such "locks" inline would probably be a bit too ugly, maybe we could instead collect them all and store them in a separate "file of locks", so to speak. Does anyone know if something like this has been tried before or am I just making up stupid stuff?

GitHub Actions could use lock files, and they would be useful specifically for composite actions, but they don't really solve the full problem at hand.

First, many GitHub Actions use a Docker image to do their work. This runs into basically the exact same problem in container land. So even if I pin the commit hash of an action, if that action doesn't pin the digest hash of its container image, then it's still ultimately a mutable reference.

Second, lock files don't protect against officially distributed malware when you intentionally update your dependencies. The lock file is designed to produce a repeatable build, and that does protect against a lot of supply-chain attacks when you never change anything, but you generally can't get away with never changing anything forever.

Third, commit hashes are SHA-1, and while it is exceedingly difficult to produce SHA-1 collisions through Git, it isn't impossible and it will only get easier over time. Git already technically has SHA-256 support but it's basically never used in the wild. This needs to be addressed sooner rather than later.

EDIT: The third point may not be as practically pressing as I thought. While SHA-1 is broken and shouldn't be used in new protocols, the techniques to break it faster than brute-force can be detected [1], and GitHub already rejects commits that are flagged by the detector [2]. I still think Git should more aggressively use a better hash, and should also be designed to migrate to new hash algorithms more easily in the future, but at least there are mitigations against attacks.

[1]: https://github.com/cr-marcstevens/sha1collisiondetection

[2]: https://github.blog/news-insights/company-news/sha-1-collisi...

I always felt that Gitlab CI was a lot more understandable. But in Gitlab CI, just as in Github Actions, you're usually running some container. And aside from the container you're also running some globally defined actions.

That's the most obfuscated part for me, the globally defined actions that can belong to any organisation in Github.

In Gitlab it was at most a globally defined git repo with templates, but you could somehow understand it better.

A better solution would be to use something like Forgejo Actions, Woodpecker CI, or Drone for a self-hosted, private setup and avoid the dependency on GitHub Actions altogether. Some of these self-hosted solutions have compatibility with GitHub Actions syntax, so the workflow would be more or less the same while having less risk for these type of attacks.
Add Gitea to that list.
I've moved away from Gitea and suggest others do the same given the controversy in the project's management.
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
You can use [ratchet](https://github.com/sethvargo/ratchet) to manage your GH action pointers.
Right, but lets get real here: GitHub Actions is fundamentally insecure. You are blindly trusting upstream libraries and GitHub itself to respect and protect your secrets.
Oh man, this doesn't sound too different from all those people (definitely not me) with hundreds of npm libraries in their dependency tree..
@actions/upload-artifact brings in 2 million bytes of javascript code to upload a file to the job artifact list.
  • r3tr0
  • ·
  • 3 weeks ago
  • ·
  • [ - ]
we have a tool that does this and more...

it is as easy as running the action and writing a SQL query.

https://yeet.cx/blog/audit-actions-runner/

can also play with it using:

https://yeet.cx/play

GitHub's own official tutorials use tags instead of full commit shas. What a mess.
[dead]