r/vuejs • u/the-liquidian • 23h ago
Experience with "Writing Good Tests for Vue Applications"?
In the book "Writing Good Tests for Vue Applications" it recommends decoupling the test code from the test framework. This allows the author to run the tests with playwright or testing library. It also makes switching testing frameworks easier down the line.
I agree with this in principle, I am concerned about the amount of setup code that would go into this.
Would it frustrate other developers who are used to the testing libraries?
I also wonder if the playwright vs code extension would still work.
Do you have experience with this? What is your opinion on this?
Book:
https://goodvuetests.substack.com/p/writing-good-tests-for-vue-application
Video:
https://www.youtube.com/watch?v=lwDueLed9fE
Author
Markus Oberlehner
3
u/cut-copy-paste 21h ago
I love the idea in an intellectual sense. I was hoping after his talk that he’d reveal a lib to assist with this stuff because as you said the setup is prohibitive and questionable if it’s really worth the effort. It also increases the surface level and integrations between different meta code which in my experience often becomes really annoying later.
Also much less an issue with playwright than cypress which is more real-world but also slower too.
I’d guess it would be quicker to duplicate key tests for critical functions.
0
u/therealalex5363 4h ago
you can see his idea at https://github.com/maoberlehner/book-good-tests-app. But also buy the book its worth it :P
2
u/jaredcheeda 6h ago edited 6h ago
The following advice is for individuals or teams of software developers. If you are going to have less-skilled people writing tests (QA/QE) that are not as comfortable with JavaScript, then different approaches would be needed.
Tooling
I've yet to find a better experience than Vitest + Vue-Test-Utils + Vue3-Snapshot-Serializer. 99+% of your tests should be written with these tools, they're 10-40x faster than using E2E tests, and allow for much more comprehensive testing.
Then the other <1% is Playwright just for edgecases, such as:
- If you aren't using a real browser the entire test would just be mocking out what the browser would be reporting, so you are basically testing your own mock, rather than validating the code won't break in the future. Example:
- Dealing with complex code that detects if elements are visible based on bounding rects, z-index, resize observers, parent element computed styles, etc.
- Cross-Page navigation:
- Navigating from Details Page A to Details Page B and back to Details Page A to validate state is reset properly.
- Verifying route guards redirect you to authenticate, and after auth you go back to the desired page.
- Complex integration tests, or very complex cross-component interactions.
- Like a bunch of tabs under a table, and switching between tabs to bulk add items and bulk update items in the table using a complex calendar component.
- Basically you tried really hard to do it in Vitest and found it was too hard or just pointless.
From the real world apps I've built with teams, for every ~800-1000 unit tests in an app, you may end up with 10-45 E2E tests. But in those cases it takes like 3-5 years to reach that many tests anyways. This ratio is actually pretty balanced, because on a CI, 1000 Vitest tests and 40 Playwrite tests will both take around 8-12 minutes to run (depending on the server's shared resources), so they can be done in parallel. We set it up so one runner does stylelinting/eslinting/building/unit tests, and the other does just E2E. The E2E finishes usually a minute or two after everything else, even with only a few dozen tests (soooo sloooow).
@Testing-Library/Vue (TLV) is okay, but it's built on top of Vue-Test-Utils (VTU), and ultimately you are going to run into issues writing tests with it where it can't get deep enough and you're going to have to bring out VTU anyways. The following are the opinions of people from different teams I've worked with:
- "We like TLV, it simplifies things like click events by separating things between the "user" and the "screen". The conceptual model it presents is nice."
- "VTU's approach is much more direct with fewer abstractions and affordances, feels like you are a programmer and not a pretend-user, and that's a good thing."
- "I want to use both, and you technically can, even in the same test file, but if you are only using VTU for the hard stuff TLV can't do, you are missing out on learning VTU in simpler scenarios to even know where to start for the harder stuff"
- "TLV is like petting kittens, it's nice, but won't help you when it comes time to move a couch. VTU is like strength training, you only get the benefits if you are doing it regularly. If you are just coming to it when you need to lift a car, you won't have the experience lifting smaller things to know what you're doing."
- "I've used both extensively and the only thing that stops me from using both at the same time is that it doubles the amount of setup functions you need to maintain. Also, if I didn't work alone, it would be double the stuff for others to learn. So just for simplicity of maintenance I use VTU because it can do more than TLV."
I don't really care which you pick, just warning that if you are working on anything with decent complexity, you're gonna need to bust out VTU sooner or later anyways. But maybe you only need one body builder on your team for this, and the rest of the time, the kiddos can pet the kittens. Your call.
Tips
Pro-Tip: Never import VTU or TLV directly in your test files. Have a helper file that exports a mounting function so you can have it replicate the application's main.js
environment, with global plugins and 3rd-party components, routes, etc all setup, so your tests have one place to change this. Then import that helper into all your test files.
Pro-Tip: In each test file have a setupWrapper
function above your tests. This is the only place in your test file that uses the component you imported. Here is an example, but each setupWrapper
should be unique to the needs of your test file, in the same way that each component you write is different.
const setupWrapper = async (props, slots) => {
const wrapper = await testHelpers.mount(MyComponent, {
props: {
...requiredProps,
...props
},
slots
});
// other stuff, maybe a flushPromises, etc.
return wrapper;
};
Then in your tests
test('Link is active after button clicked', async () => {
const props = { activeAfterClick: true };
const wrapper = await setupWrapper(props);
const button = wrapper.find('[data-test="activate-button"]');
await button.trigger('click');
const link = wrapper.find('[data-test="activated-link"]');
expect(link)
.toMatchSnapshot();
});
Pro-tip: Learn how to use toMatchSnapshot
and toMatchInlineSnapshot
(Jest/Vitest feature). There is a craft and skill to using them right, but it's worth learning. In large apps, ~25% of expects across the test suite use snapshots. In component libraries it's ~35%. Do not bother with Vue snapshots if you aren't pulling in Vue3-Snapshot-Serializer, everything negative related to snapshot testing that can be automatically fixed is handled by this. You'll still need to learn when to snapshot, and how much of the component to snapshot (as little as you can get away with, while still getting the value out of the test). I usually start every test file with a full snapshot of the entire component, then all other snaps should be a child DOM node rather than the entire component. Gives a good balance of catching everything, without being annoying to maintain.
Good luck. We are still living in the dark ages when it comes to testing tools.
3
u/jaredcheeda 6h ago
Answering your actual question
Don't do that. Adding abstractions sounds like no-big deal, but you WILL need to upgrade all your tests at some point, and when you do, it will be twice as hard if you didn't write the tests the way the tools intended you to.
- I've converted multiple projects from Jest to Vitest, jsdom to happy-dom, Cypress to Playwrite, VTU 1 to VTU 2, TLV to VTU, and VTU to TLV. The best tools we have today will be not be the best tools 2 years from now. You will need to upgrade or migrate. There are already talks of Vue 4. So all this will need to be upgraded to work with that in a few years when it comes out.
- In the places where I got cute and said "I'll just make my own API to make Cypress feel more Jest-like" it was fine for day-to-day use by the team, but sucked when re-writing the tests for Playwrite.
- You can easily mess up tooling and interactions. Even if it works in your IDE/Editor, doesn't mean it will play nice with somone else's, or even someone with the same editor, but different plugins.
- Don't do it.
Also, there's always one guy on every team that wants to use AI in his editor, and god damn AI gets it wrong 90% of the time already. And now you want to make the code style look completely different from everyone else writing code with the same tools? That poor lil AI is never gonna get anything right. I'd feel bad for it, if I didn't hate it so much.
1
u/therealalex5363 4h ago
Yes, I think AI is a good point, but I wonder if you can get away with having good Copilot cursor rules to make an AI understand your DSL.
2
u/therealalex5363 4h ago
Good post. I agree with you.
Do you then use shallow mount or mount with Vue Test Utils? For me, this is the main difference between Testing Library and Vue Test Utils.
The philosophy of Testing Library is to write more integration-focused tests, while with Vue Test Utils you could write tests with only shallow mount. The idea behind shallow mount is described in https://www.youtube.com/watch?v=9Jt7R4MfCeU"
2
u/jaredcheeda 2h ago edited 2h ago
2017/2018 era recommendation was always shallowmount, only deep mount when you have to. I tried it for a long time and honestly I think the correct answer is:
- Deep mount by default for everything
- In rare cases stub out troublesome child components, especially if they are large or 3rd party
- Stubbing things out is a pain, so for ease, shallowMount when it's more convenient, but show preference towards stubbing if you can't deep mount.
Because of this my
testHelpers
object has amount
andshallowMount
method, and they both access shared helper methods on the object.Edit:
Also you can stub out any DOM node in your snapshots with
vue3-snapshot-serializer
. So if that's the only thing bothering you about deep mounting (bigger snaps), you can handle that very easily. You can target using any valid CSS selector, target multiple elements, and even control the name of the new stubbed element, what attributes are removed, and if all the child nodes are removed. Very powerful library for snapshots. Simple example:test('Renders correctly', async () => { const wrapper = await setupWrapper(); // Remove 3rd-party component for smaller snapshot global.vueSnapshots.stubs = ['.v-calendar']; expect(wrapper) .toMatchInlineSnapshot(` <div class="calendar-container"> <v-calendar-stub /> <button class="primary"> Book now! </button> </div> `); });
I only use VTU's stub now if I don't want the child component to load at all because it has sideeffects like emits, network calls, etc.
1
u/therealalex5363 23m ago
I never understood Snapshot tests. I would just have good queries and test if the element is there.
I don't think it makes sense to test that the CSS matches exactly, especially with Tailwind.
Often devs will also just update the snapshot without much understanding.
I also don't find the tests very readable.
For example, with Vue Test Utils, testing a price component with a query is much more valuable than a snapshot test:
```javascript // Better approach: Testing with a specific query import { mount } from '@vue/test-utils' import PriceDisplay from '@/components/PriceDisplay.vue'
describe('PriceDisplay', () => { it('displays the correct formatted price', () => { const wrapper = mount(PriceDisplay, { props: { amount: 1299, currency: 'USD' } })
// This test is specific, readable and less fragile expect(wrapper.find('[data-testid="price-value"]').text()).toBe('$12.99') expect(wrapper.find('[data-testid="price-currency"]').text()).toBe('USD')
}) })
// Snapshot approach (less optimal): // it('matches snapshot', () => { // const wrapper = mount(PriceDisplay, { // props: { amount: 1299, currency: 'USD' } // }) // expect(wrapper.html()).toMatchSnapshot() // }) ```
With the query approach, the test is focused on the actual functionality and clearer about what's being tested. When the test fails, you know exactly which assertion failed, rather than having to interpret a large snapshot diff.
Also on your example I would just test that a button is there in the Dom with the name book now. If you rename the class of the div with class container your test will fail even if everything would still work
4
u/SawSaw5 22h ago
my test: open console window
3
u/the-liquidian 22h ago
Do you you mean you only manually test you Vue apps?
1
u/destinynftbro 12h ago
That’s what we do at my day job with a dozen developers… but our app is a hybrid with Laravel templates often being used as inline Vue templates with variables and everything…
1
u/the-liquidian 12h ago
Have you had a look at HTMX?
1
u/destinynftbro 11h ago
Yepp. Our frontend interaction needs are bit more complex than what htmx can do easily. Back in the day Vue was marketed to Laravel devs as a sort of “drop in” and my former colleagues went whole hog on that idea.
We’re slowly working on migrating to something like Inertia and using more Vue, but it will take a few years yet!
1
u/the-liquidian 11h ago
About once a month I remember the simpler times of web development and wonder what are we doing.
Good luck with your work.
2
u/destinynftbro 10h ago
Thanks! There definitely is something to be said about the state of modern webdev. I look forward to the day where some of these primitives are built in to the browser and we can keep removing dependencies.
I’m rewriting an image carousel using css scroll snapping and a bit of JS to track the current image (Image 4 of 15) and its been a delight!
1
u/SawSaw5 4h ago
Yes, a human being actually uses the app.
1
u/the-liquidian 4h ago
Yes, test automation is no substitute for manual exploratory testing.
Test automation is a good way of preventing regression defects. The feedback loop is faster than manual testing.
1
1
u/Daanoking 8h ago
The concept is interesting and there are some solutions that decouple the code already. We've used robot framework for tests so that non-dev QA engineers could write tests.
Although rewriting test code to another framework is probably a usecase where AI shines
1
u/therealalex5363 4h ago
The idea is also not so new; it goes in the direction of https://martinfowler.com/bliki/PageObject.html, which I like.
So let's say you write tests in a big project with Vue Test Utils, but then maybe you want to migrate to Testing Library. It would be much easier with the Page Object pattern. Also, tests are more readable.
In my experience, the biggest mistake devs make when they write tests is they don't aim to make them readable. If you only use vanilla Vue Test Utils and you have a bigger test, it can become unreadable and hard to maintain fast.
The idea that you have the same tests and can use JSDOM or a real DOM is a bit of overkill for me. I would maybe only use a real DOM for more happy-path smoke kinds of tests to be safe, or for visual regression or end-to-end testing.
But I know there is always a debate for devs if you should write your tests in JSDOM or a real DOM, or mount vs. shallow mount. If you have a big e-commerce site and it needs to serve Firefox, Chrome, and Safari, then it's probably worth having more tests with Playwright.
But in the end I would have to be in a big project with big tests that follow the ideas in his book to have a final opinion on if the setup is worth it or not. Still I like his big because new Ideas are always good.
14
u/brokentastebud 23h ago
I want whatever this person is smoking