UI, Unit, Integration, Security Tests required?
My friend Jakob and I are currently developing a finance app:
https://github.com/The-Creative-Programming-Group/finance-io
We’re currently having a discussion regarding testing.
Some of the points mentioned are related to this video by Theo:
https://youtu.be/nSOmL2DxzbU?si=gbt6MJ4wTGPzibii
And also this blog post from PostHog:
https://posthog.com/product-engineers/testing-in-production
The core question is:
Is it meaningful to integrate tests into an app that hasn't been released yet?
My standpoint is:
As soon as we acquire users, we hold a level of responsibility.
Jakob, on the other hand, doesn’t want to add tests. He prefers testing in production.
Why do I believe this approach is problematic?
Especially for a finance app, errors might be detected too late, and could lead to:
Sensitive data being exposed
Financial operations failing or behaving incorrectly
Loss of user trust
Jakob's argument is:
"As long as we don’t have any users, we don’t need tests."
But I believe that releasing version 1.0.0 without tests is risky, since the goal of that version is to:
Start attracting real users
Be used for marketing and outreach
This means the app should already have a baseline of stability and reliability.
He also argues that we should quickly respond to user feedback.
And I agree with that.
But I don't think that contradicts my view. We can still:
Move fast
While having basic automated tests in place to prevent regressions and critical issues
I also believe that an app like this requires:
Planning, defining clear requirements, and
Thinking through features before rushing to ship
This might lead to slightly longer development times, yes.
But it greatly reduces risk and ensures a more secure and higher quality application.
In summary:
Testing early is not about slowing down.
What’s your opinion?
GitHub
GitHub - The-Creative-Programming-Group/finance-io
Contribute to The-Creative-Programming-Group/finance-io development by creating an account on GitHub.
Theo - t3․gg
YouTube
I Only Test In Production
SPONSORED BY POSTHOG
Unit tests, integration tests, staging tests, what about testing in production? Great question. I think we should do it a lot more
Thank you PostHog for sponsoring the video!!! Link to blog post
https://posthog.com/product-engineers/testing-in-production
Check out my Twitch, Twitter, Discord more at https://t3.gg
S/O Ph4...
How to safely test in production (and why you should) - PostHog
At PostHog, we test in production. There are many misconceptions about doing this. It does not mean: we commit to main every time we make a change…
8 Replies
Note from the post hog post. They do test. Testing is done locally and done again in production. What they don’t do is test in a separate testing environment ( because they found it too difficult to keep a good test environment running and that it didn’t find enough issues to make it worth the cost).
Thanks :D What do you think about our Szenario?
But is writing tests the best way for an app with zero users and not even a lot of code
It's just annoying me that all my important time (I'm not working full-time on this, for everyone it's a side-project) is going away for tests, and meetings, and documents, and plans, etc. instead of the important things
This is important tho ;)
Well, you should test as you code. Writing tests should be done as you write the code. It is part of the process for applications that you actually expect to have users for , not an afterthought.
There is the old saying about if don’t have time to do it right in the beginning, what makes you think you will have time in the future to fix all the things you didn’t do right the first time?
If you find yourself working on everything but code, welcome to business. I assume you are building this application with the intent of making money in a business. If so, then this side work is required (much of it is very important once you start to grow). If not, then just do the fun things.
What important things are you not doing? Untested, undocumented, unplanned coding?
Before you have users, prod is test. It is not until you have real users that your environment is really production.
Another saying is: Everyone has a test environment. Some people build a separate production environment.
But how do you know what you is the right thing?
Maybe you waste your time doing all this stuff "right" but in the end it turns out it was for nothing cause you didn't actually count something in that you couldn't know in the first place
So maybe then it's more important to build something then to build it the best way possible or with a gazillion tests
You understand what I mean?
There are always trade offs. Building the best possible thing is often a circle jerk. But so is building something that your customers will reject as a piece of junk.
You only get one chance to make a good first impression.
Don’t build a gazillion tests. Build the tests that matter. Don’t build unnecessary features. Build what is required to impress (and provides a useful service to) the early customers. Then do a mix of making customers happy while making new stuff.
Also, don’t be afraid of failure. Some of Theo’s early products didn’t last/make lots of money. He had to shift/iterate until he found something that worked.
All of other stuff might not work for this product, but it will be good experience and will be useful for the next product.
Yes, if you bury yourself in busywork and miss a key feature because of it, then that is an issue. So the solution seems to be to not bury yourself. Don’t take the comment that you should write tests to mean that you must obsess over them to the exclusion of everything else. Take it to mean that they are important as PART of the development process, which is PART of the business process of selling a product or service.
If you just want to code, either do it for fun, or get a corporate job that is mostly coding.
If you don’t know what the right thing is, there are consultants that can help with the process. But you will have share lots of details for them to be useful.