How do you test pull requests beyond just reading code? (Research)
Hey developers!
I'm researching how teams review and test pull requests, especially when you want to actually try out changes rather than just read the diff.
Looking specifically for folks who:
- Work on teams that review pull requests
- Deal with multi-service applications (backend + database + frontend, etc.)
- Sometimes need to actually test/click around to validate changes
Quick questions:
- Do reviewers on your team actually spin up environments to test PRs?
- How do you handle testing changes that involve multiple services?
- What's your biggest friction when you want to see changes working live?
- Has AI code generation made you want to test changes more thoroughly?
This is pure research - exploring building a tool but want to understand the real problems first.
Also fine to DM me if you want to discuss your current setup.
14 Replies
Cloudflare pages and vercel do this automatically, at least for the frontend, but not sure about multi service. If you were using a database with different environments like convex you could have a dev database and connect to that in the testing application. I have used tools like coderabbit to review changes and it tends to pick up a lot of issues (better than gh copilot). I am primarily building personal projects with minimal collaboration apart from dependabot PRs so I don't really have anything more to add
thanks, yea, thinking of more fullstack applications with database, backend, potentially multiple backend services, frontends.
vercel is great for preview envs for frontend btw
before merge to main branch, all prs should be point to dev branch first. Then check on test server with real prod DB as well. All actions should be done manually. If the logic complex, QA team need to check in several cases. Like pr created in several branches and each of them are deployed in unique server. Then one by one merge to dev branch with staging first, if it failed, revert to old working code. This is what our team currently doing. Once they found out any new issue, they create ticket and link to prev updates link. And reproduce flow as well. So that dev can understand easily what will be failed.
Also before in pitchdays, all fixs should be mentioned. There is not any tools. Just human check is reliable.
bots can check only code side issue, type mismatch... common issues.
If the PR touches frontend code, I'll checkout their branch and run the FE locally, connected to real prod backend and test manually.
If the PR touches backend code, depending on the service sometimes we have a way to run it locally and connect it to the rest of the system for manual end-to-end testing. If so I'll do that. If it's BE code that's hard to run locally then we just rely on unit tests / CI checks.
Once code is merged it's deployed to a dev environment, then staging environment the buisness next day, then production the day after that.
I work at big company where we have lots of end-to-end tests and manual QA that happens before a prod deploy, and also really mature systems for gradually rolling things out and heaps of alerts/monitoring and very good tooling for rolling things back when something goes wrong so it's usually not that stressful
we clone production db and removed sensitive infos from there because of pop. And deploy on staging server with that db. QA and reviewers are on there fully. Once passed, it will be deployed on dev branch, and then production as well. It's common infra
everyday logging is also important. so can be roll back to stable version if unexpected problem happened on production
Are you also using github actions? Jest? Cypress? etc
git action
also cypress
interesting, seems like your are both on pretty big companies. how many engineers are there working?
Is your product spread out between different repos, or do you use monorepo?
Can there be two staging / preview environments up at the same time? Or only one?
Are there other people other than engineers/qa that uses staging?
how many? it's up to project size. It doesn't matter.
We not use monorepo. for big projects, it's not good to manage whole code in one repo with across web/mobile/desktop ...
cause of ownership problem, ci/cd problem, reviewing system...
we run multi env as many as possible on per PR
it's good for testing before merge to prod version
Also staging is not just for qa, pm only. Other designers, devs, someone else who can support and give feedback to us.
I work at a company with over 5000 employees and over 2000 engineers. Everything in one giant monorepo, but still a service based BE architecture with independently deployable pieces.
Dev environment (where code gets deployed very soon after merging to monorepo) is used exclusively by engineers. Staging environment (where code gets deployed about 1 day after deploying to dev) used almost exclusively by engineers and QA.
but in this case, all users have full access of whole source code. Or do you use any external tool to manage ownership? git doesn't support natively it
Yep, all engineers have access to all the code, but almost no engineers can actually push directly to master or to deployment branches - to get new code into master and deployed you must create a PR and then use our custom tooling to merge it, and this tool will refuse to merge unless the code has the right reviewers and all CI checks pass.
We have files inside the codebase which indicate which engineers own which code, and trying to merge a pull request that hasn't been approved by at least one owner of every changed file will fail
we already using this branch rule too.
But in terms of source expose, individual dev don't need to have full source of application. So we split ownership of them per repo.
We're so strict in source code manage.
I find it super nice to be able to always read the code you're trying to integrate with. I'd be sad if my workplace introduced annoying source code access controlls 😢
For where I work - treating source code as top secret+super sensitive doesn't really make sense since like 70% of it is client-side code. Javascript is JIT, so all our client side code is visible to everyone on earth anyway - all they have to do to get it is open our app in the browser (it's somewhat obfuscated due to tsc/closure compiler/chunking/minification but would still be totally possible to extract)
I suppose it makes sense to be very strict with source code management if the business success depends on very unique secret algorithms, like at a trading firm or something.
Access to source code can also be useful for someone trying to launch a cyber attack... but it only helps find the vulnarabilities that already exist - if the company has a whole team deditacted to security then being secretive about soruce code doesn't add that much protection.
Where I work, we do have a whole team foucsed on security, and the thing which makes our product better than competitors is superior design+UX which by its nature has to be exposed to the public regardless