Why 1600 Testing Scenarios for WordPress is Not Nearly Enough, with Juliette Reinders Folmer

Episode 2 May 09, 2023 00:52:10
Why 1600 Testing Scenarios for WordPress is Not Nearly Enough, with Juliette Reinders Folmer
Within WordPress
Why 1600 Testing Scenarios for WordPress is Not Nearly Enough, with Juliette Reinders Folmer

May 09 2023 | 00:52:10

/

Show Notes

Learn all about how Juliette Reinders Folmer uses 1600 testing scenarios for just one little thing inside WordPress. Juliette is one of the big forces behind getting the code inside WordPress tested and up to spec according to PHP standards. Her work for the WordPress Project is fueled by wanting to leave the world in a better place than how she found it, and oh my word, does she ever!

Juliette Reinders Folmer is an opinionated, passionate busy-body with a prolific portfolio of contributions to various high profile open source projects over the years. She is co-author of PHPCompatibility, initiator of PHPUnit Polyfills and created the PHP Cheatsheets]. Juliette regularly speaks at conferences and is always happy to discuss how we can all contribute to changing both the dev-culture as well as our code, to be _better_.


Juliette is self-employed and based in the Netherlands.

View Full Transcript

Episode Transcript

[00:00:08] Speaker A: Welcome to Within WordPress, our podcast about everybody inside of WordPress that you should be knowing, should be talking to. In today's episode, I am welcoming Juliet, one of the strengths behind, in my opinion, the optimization of WordPress from a different perspective than just performance, but making the whole project better in many, many facets. But she'll tell everything about that herself. Juliet, welcome, welcome. [00:00:40] Speaker B: Thank you for having me. [00:00:42] Speaker A: Yeah, of course. And yeah, why don't you introduce yourself and tell us who it is we have on the podcast. [00:00:52] Speaker B: My name is Julia Durandis Fulmer, and don't try to pronounce my surname if you're not there because you'll break your tongue. Other than that, people sometimes ask me what do you do? And give me the elevator pitch. Excuse my French, but the short version is I do whatever the fuck I like and I hope I get paid for it once in a while. [00:01:16] Speaker A: That's a solid strategy. I think that's a good one to start. So what is it you like? What is it like you like to do? What is it you do within WordPress? Maybe start even there? [00:01:30] Speaker B: Well, I think it's more everything. I like to leave the world a little better than when I came in it. And that can be in all sorts of small things like recycling things which you otherwise might have thrown away, but it's also in making the web better and making it I do a lot of things behind the scenes which enable other people to make the web better, which then enable publishers to publish in a better way. So it's like a domino's where when you make sure that certain things at the back are enabled in a way to let other people perform optimally, then the whole domino range will perform better. [00:02:19] Speaker A: Yeah, the cascading effect, that's a great example of somebody doing a lot of stuff and a lot of good stuff for the project and being very modest in what she does, because you just explain what you do without actually telling us what you do. [00:02:41] Speaker B: To be fair, and this may sound strange to some people when I look at what I do, I don't actually do that much for WordPress. I do a lot of around WordPress, but not directly on the cord. I do touch the cord code and I do make, fixes and review BRS, et cetera, but it's only a very small part. The things I do, for instance, an external dependency calls requests. I'm one of the maintainers now because in practical terms been abandoned by the original maintainer. But WordPress depends on it. Yeah, other things another external dependency, PHP Mailer, is still maintained by another external maintainer. However, their test suite was definitely in need of some 1011 care. So last year I spent a couple of weeks getting their test suites to a much higher level, which sets them up to be compatible with PHP Nine when that comes around. So that by the time WordPress needs to get ready for PHP Nine, they can again, those dominoes, getting all the little pieces in place so that by the time WordPress needs something, it can do it. [00:04:05] Speaker A: Yeah, and I think this is the beginning of where you started explaining what it is you're doing. You do a lot that has to do with testing, that has to do with code standards, that has to do with making sure the project in terms of code is up to spec. If I were to give a TLDR, would that be the correct version? Am I missing? Because you just mentioned you're maintaining two, probably more libraries that also help doing stuff inside of WordPress because it depends on it. [00:04:36] Speaker B: But I don't know how many repositions I'm maintaining got commit to. [00:04:43] Speaker A: Probably more than I'm comfortable with knowing. [00:04:46] Speaker B: Yeah, more than I'm comfortable with. [00:04:48] Speaker A: Okay, so it's time to scale back a bit. [00:04:53] Speaker B: I challenge people when they want to give me commit rights nowadays, I don't accept them straight away and I prefer not to accept them a lot of the time. [00:05:03] Speaker A: Yeah, I understand. So for those who are still a little bit unsure what it is you do exactly. So what would be the projects that would have the largest impact on those just using WordPress to create content? Is there anything that you do when you stop doing it? What would they see or wouldn't they see? [00:05:30] Speaker B: It's not one thing. Again, it's this cascade, as things are at this moment. WordPress runs fine on PHP 74, runs up to a certain point for the end user. It runs fine on PHP Eight Barring plugins, which may not be compatible, but come PHP Nine, we have a different problem. [00:05:59] Speaker A: Yeah, I would say we have a little bit of problem at 8.1 still. [00:06:04] Speaker B: No, we don't. The thing is deprecations, what people can see now in WordPress still are deprecation notices. And deprecation notices are basically your action list for the next major, which means PHP Nine, they don't need to be solved at this point. But for people, that's a good distinction, difference between a deprecation notice and an actual error that can be confusing. [00:06:33] Speaker A: That's a good distinction to make. I think a lot of people don't really understand what happens at that layer. When is something a notice, when is it a fatal, when is it a deprecation of that? Yeah, obviously, but those three are introduced when you bump a PHP version and you play a part in that somewhere. So what is the part that you play? So what if I am on 7.4 and I go to eight? What is it that you have done along with other contributors, but what is it that you have done specifically to make sure that bump is as smooth as possible? [00:07:12] Speaker B: Well, I've fixed quite a few issues in the code itself. [00:07:18] Speaker A: So inside WordPress code fixes for PHP upgrades. [00:07:24] Speaker B: Yes, but also I well, contributed to in a later stage built tooling for WordPress. Well, not for WordPress, but which is now used by WordPress to allow the tests to run on new PHP versions, because WordPress was still struggling with how do we run the tests on PHP Eight? Because the test tooling used PHP Unit made some breaking changes and now we can't easily run on multiple PHP Unit versions, which you need for PHP Eight support and for testing on PHP Eight. So one of the things I built, and that's sponsored by the Yosh company, is the PHP Unit polyfills. That is a library which allows you to write your test for the latest PHP Unit version and then run on older versions without having to do anything else. [00:08:25] Speaker A: Is that what the library does? The polyfill part is filling out the part, so that whatever, if you're writing a test for 8.1, it backfills it to the older versions. Is that how I should interpret that? [00:08:38] Speaker B: Interpretate that fills the test functionality, which was so in PHP Unit Nine, there were a number of assertions added or renamed or changed for certain parts of that. It meant that you'd have to have lots of if else in your code to be able to run the tests, and you need to do that in a lot of places. The polyfills make sure you can just use the new assertions as per PHP Unit Nine. Then the polyfills will take care of all the test functionality which is needed to be able to run those tests on PHP Unit Five or four. [00:09:26] Speaker A: Dear Lord, I hope we're not doing four still. [00:09:29] Speaker B: No. I'm currently working on the Polyfills version 2.0, which will allow the same, but then for PHP Unit Ten as the target fingers. Ten was released in February. [00:09:42] Speaker A: Yeah. So would it be a fair assessment to say without you we wouldn't have as modern of a WordPress base as we currently have? Or is that too much credit? Or too little even? I don't know. [00:10:02] Speaker B: I expect that if I would step down, other people would step out. [00:10:05] Speaker A: Sure, let's assume they're not. [00:10:12] Speaker B: I would love to see WordPress modernize more. I think we need to modernize more. I think we need to step up our game lead by example. Also towards plugin developers and on the one hand with modernizing the code base, on the other hand, with having a more comprehensive test suite. Lead by example. As WordPress core to show the community what is needed to get ready for PHP Nine. Because it's going to be a nightmare otherwise. PHP Nine is going to be an onslaught. The way things are at the Migrant. [00:10:46] Speaker A: More than previous versions sorry, more than previous versions of PHP, like eight? [00:10:52] Speaker B: Eight one, absolutely. [00:10:54] Speaker A: Okay, interesting. So would you say we're already behind or are we on track or about ten years behind. How many years behind? [00:11:05] Speaker B: About ten years. [00:11:07] Speaker A: About ten years? [00:11:11] Speaker B: Well, 10, but yeah. [00:11:15] Speaker A: 10. [00:11:17] Speaker B: Yeah. No, I mean, realistically, if we were to have a hope of catching up before PHP Nine comes out, we should have started three, four years ago. Oh, wow, we haven't. And any efforts and any discussions I try to start around this are being quickly killed off. [00:11:39] Speaker A: Which, by whom or by what? [00:11:43] Speaker B: There are certain people in the WordPress team, core team, are not a fan of any type of modernization and basically believe PHP Four code should still run on PHP Nine. [00:11:58] Speaker A: And the reality yeah, the backwards compatibility crowd, essentially. [00:12:04] Speaker B: Yeah. Except there's between backwards compatibility and fixing bugs. The thing is, what I see a lot happening in WordPress is let's just hide this PHP error, let's just add a TimeCast or just to make sure, correct type, except that's not actually helping anything. No hiding errors instead of fixing them. And most of the time they should not be fixed in WordPress core, but they should be fixed by in a plugin or in a team which is doing something wrong. WordPress has documented functionality. If a function, a WordPress function says, I expect a string pass, it a string, and if you're not passing a string and you get an error, notice that's on you, you need to fix that. [00:12:50] Speaker A: Yeah. [00:12:52] Speaker B: What WordPress could do to improve that experience is of course, add type declarations. But to have type declarations, we need PHP Seven out. It could mean fatal errors, and that may be a bit too harsh for allowing people to gently upgrade. [00:13:11] Speaker A: There's nothing gentle about a fatal would. [00:13:15] Speaker B: Be add type checking to every single functioning core with notices to tell developers to get their shit together. [00:13:25] Speaker A: Yeah, that's a huge task. [00:13:29] Speaker B: It's huge. And it also means we need to add tests for all of it. [00:13:33] Speaker A: So out of curiosity, are you familiar with how other CMS do this? Are they just, okay, go ahead. We're breaking whatever backwards compatibility or however, whatever construct you want to call it, but we're done. Is that kind of what Drupal does with says? We have version seven, and if you want to upgrade to eight, you're just going to have to put in the work. Is that the sort of attitude that we should adapt? [00:13:59] Speaker B: I'm not saying that's what WordPress should adapt, but WordPress should stick to the documented functionality and not keep expanding the functionality to cover every single type and try to handle everything. It's like you're saying, okay, we have a blender and we just put in everything, and then it's still supposed to be drinkable what comes out of it, except if you put in the wrong ingredients, it's not going to be very pleasant to drink anymore. [00:14:30] Speaker A: It may be fluid, but it won't necessarily be pleasant to drink. Yeah, that's a great analogy. I think I can use that more often. The thing that I always find interesting is the and I've known you for years, right? So I know the stuff that you've had to do the stuff, you had to force yourself into, get other people to align with you just to get the ball moving forward. I know that's been a task that's been resting on your shoulders for digging back for as far as I can remember, at least seven, eight years, maybe longer even possible, I don't know. Yeah. [00:15:15] Speaker B: It'S the making things better, and doesn't matter whether it's WordPress or anywhere else, I always will strive to make things better. [00:15:23] Speaker A: Yeah. Oh, clearly. Because that oozes out of everything that you do, you have an incredible drive and you show it as well in what you do to make things better. I think for someone listening to this or watching us, who has no idea of they probably understand that WordPress needs PHP, because in the requirements it says so what is it that they can do to understand the problem better in terms to help move along the project? I'm trying to look at a perspective of you are very skilled in what you do and with your skills, you're helping the webtress project itself in direct and indirect ways. For someone not really understanding or grasping the full concept of PHP and how that's compiled and how that does its thing, but wanting to help this particular way of improving WordPress forward, is there a way, is there a way for someone to push you along, or for. [00:16:41] Speaker B: Someone without any depth skills or credits? It is hard probably to contribute to this, but one thing they could do, just as a site owner, you can turn on error logging and you can set the error logging to a different level than what's used on the front end of the site. So on that of the site, you don't want to be bothered with notices and deprecations and everything. You can try to set up error logging, so those things do get registered in an error log and in a text file, basically, and go through those every so often and report the deprecations you see, especially the ones you see multiple times to the plugin owners. And if the core one decor, but most of the time it'll be a plugin. Yeah, the thing is, you need a backtrace for this because a deprecation notes may come from core, but be thrown by core because a plugin calls a function in core incorrectly. So you always track traces and you need to report it to the right place. So report as many issues you see. And I know this is not an easy task for someone who's not a developer, but if you really want to help reporting those issues, to plugin developers, to team developers, to try and allow them to get the most relevant information, to get their shit together, will help the project in general. [00:18:27] Speaker A: That's a good one. So the backtrace when you said that, I remember the first time I learned about that, this is years ago, I saw an error. And I was like, WordPress is throwing an error, so there's an error in WordPress, but that can't really be right. Why would I be the only one having that error? I didn't get it and I don't know who pointed me in the right direction, but someone said no, no, you need to do a backtrace first to see what actually is causing this. I'm like, no, but WordPress is telling me to do no, no, do a backtrace and see what happens. And figured out how to do that, turned it on. There's a silly plugin that's just doing something not entirely correct. That's a great example of where you can have an impact without actually knowing how to code is searching for error logs. And I'd like to add to that. I've seen situations where hosting companies actively turn off error logs of any kind, or if a piece of code is taking too long to execute, they will just softly kill it off. You'll never see it, you don't see the fatal nothing. And you'll think, hey, my code is working fine. But when you switch to a host that then does no limitations on any error logging whatsoever and you use the same code, same PHP version, and you see all notices and stuff, you go, what's happening? What's wrong with this new host? No, you were being muffled, you were being silenced. There's a lot going on. [00:20:01] Speaker B: I have to say though, I know you also do a podcast about performance. Turning on error logging has a negative effect on performance errors. If there are no errors, it has no effect. But if there are deprecations and warnings and notices, the fact that those need to be locked, that need to be written to a file, will have an impact on the website for sure. Turn that on for a day or two at a time, analyze the logs, report everything. After you've upgraded all the plugins, after they've released new versions, turn it on again for a day or two. [00:20:41] Speaker A: That is exactly what I do for any site that I maintain that has dependency on being performant, on being fast, or especially if it's Ecommerce or WooCommerce mostly whenever you're doing a deployment. And this past weekend I gave a talk on WooCommerce, but faster, where I mentioned this as well. If you want to treat yourself seriously, you need to test every single deployment on Local first. Then when you're ready and ready to go, go to production. And then test every single aspect of that site that is vital while doing so, having everything turned on in terms of error reporting, because you'll find information you haven't seen Local, I guarantee you it happens. Local, nothing's wrong. Yay good. And we have the same blueprint, right? Local is the same as product, but it's not, it's never, it never is. [00:21:40] Speaker B: It's also about knowing how to test. [00:21:44] Speaker A: Yeah, for sure. [00:21:45] Speaker B: If you think like okay, I've upgraded something. Hey, the home page loads. Now, I've tested it, I literally mentioned. [00:21:53] Speaker A: This in my talk. They're saying if you're only testing front page, you're doing something wrong because your cart the talk was on WooCommerce. If you're not testing your cart, if you're not testing your checkout the whole thing, what are you testing? You're not, you're testing some static caching. Wonderful. [00:22:17] Speaker B: It works, which provides a contact form. You need to actually send in that contact form and check both that the sending works correctly and the email gets received and you get your information, all the things, it's all in the back end, as you expected, et cetera. And there's so many moving parts with plugins, so for every plugin, you need to know what it does and know what to test. And that is already a big task for people. [00:22:44] Speaker A: Yeah, I was going to say, I'm amazed at how few people understand what a testing means. That you need to start with testing your uncached performance, that you need to start with testing your uncached carts, obviously. [00:22:57] Speaker B: But presumes that people actually know what a cache is. [00:23:01] Speaker A: Yeah, it probably starts right there. Yeah, I know we can't expect everybody to understand all layers, and this is a podcast where we're talking a little bit more in depth in terms of PHP and in terms of testing and deployment to production and what you should and should not do. I don't expect everybody to understand this podcast as well as the one I did with Mad Madeiras, for instance. But the idea that you need to validate changes as you're pushing it forward in your pipeline from staging to production, or from local to production or whatever, but test, what is it to test? [00:23:47] Speaker B: It's not actually you need to validate your changes, that's what you need to do as a developer. As an end user, you need to validate that the functionality still works the same before and after. [00:23:59] Speaker A: Yeah, that's a good distinction. Yeah, you're right. [00:24:03] Speaker B: Yes, end user testing is very time consuming and it's very imprecise as well. I rather see that plugin developers, especially plugin developers, team developers as well, but especially plugin developers, got into the habit of having an automated test suite, and again, WordPress needs to lead the way with that test suite. [00:24:30] Speaker A: No, we don't. [00:24:32] Speaker B: Percent of the code base. [00:24:34] Speaker A: So if people wanted to test this, and that's a double, but test a test thing. [00:24:40] Speaker B: You're missing the point I'm making. Owning of a package which runs 40% of the internet, 20% of the PHP code is being tested. [00:24:50] Speaker A: Oh wait, 20%? Yeah, that's a 20. [00:24:54] Speaker B: Yeah. WordPress core has, relatively speaking, very little tests. We need to raise the test coverage, because otherwise there are changes now happening in PHPA which we cannot find with static analysis, where you need test coverage, where you need your tests to run to show you those deprecations in all those code paths which are not frequently used, but are available and are supported. So we need to see all those deprecation notice to be able to fix all those things before PHP Nine. This moment, we've been able to do that for about 20% of the code page, not for the other 80%. The other 80, we depend on people reporting things. [00:25:43] Speaker A: It's the same reason this is not happening. The reason you mentioned earlier, there's those among the core team that don't care. [00:25:53] Speaker B: For this particular no, I mean, the test coverage is getting a little bit more attention nowadays, but it's not sexy, let's be fair. [00:26:05] Speaker A: I don't know, I kind of find a testing procedure coming up with stuff that would have broken my site in one way or another. I find that sexy in one way. [00:26:14] Speaker B: I find it exciting work, I find it really valuable work more than anything because I always find bugs. Yeah, but I mean, there's a couple of layers here. On the one hand, a lot of, especially younger developers, they want to work on something exciting and new and functionality which you can offer to the end user. Writing test is not sexy, new features is sexy, so people are less interested in this. Second of all, there are tests and there are tests. You can have a test which goes through your code and shows 100% code coverage, but is a crap test because it doesn't actually test it. [00:26:58] Speaker A: It just runs through it. [00:27:00] Speaker B: Yeah, it just runs through it. Basically, it's what we call happy part testing and unhappy part testing. You can have a code which passes the test and code which fills the test set up correctly. But if the code is supposed to do a certain thing and behaves potentially differently with different inputs, you need to document that. You need to make sure that the code behaves correctly with the input type you expect and rejects all the others, for instance. So the unhappy part, making sure that the code doesn't act on something where it shouldn't be acting on. Often the forgotten part, or is tested too little. I have a function in another project somewhere where there's 30 lines of code and there's two quite complex regexes in those 30 lines of code. Yeah, I have 1600 tests just for those 30 lines. [00:28:02] Speaker A: 30 lines? 1600 lines of test code? [00:28:05] Speaker B: No, not lines, actual tests. [00:28:07] Speaker A: Oh, actual tests or even more? [00:28:09] Speaker B: 1600 tests. Holy, the last hundred tests I still found bugs. [00:28:17] Speaker A: Yeah, that's not discouraging to you somewhere. [00:28:20] Speaker B: Because I know it's better now. [00:28:22] Speaker A: Yeah, the end result is better, but. [00:28:26] Speaker B: That it stays good. [00:28:27] Speaker A: Yeah. [00:28:29] Speaker B: Yes. This is frustrating the first time you're working on, but the next time you know that everything you tried before is automatically tested. [00:28:41] Speaker A: Out of curiosity, is this a particular regex type of thing that just happens to be extremely like a wild card type of thing? Or would you say there are more cases where we would end up with. [00:28:58] Speaker B: That many tests in this particular case. Okay, for those people who are developers. [00:29:04] Speaker A: You may think your audio went up. [00:29:06] Speaker B: Different PHP coach Niffer is a static analysis. [00:29:10] Speaker A: Can you hear me still? We're back because something happened there, we don't know what, but something happened. So we just switched to zoom. So it'll look different from now on, but we're not quite sure where we left off. But I think it had something to do with 1600 generated tests based on 30 lines of code. Like I was saying right before we started recording again on Zoom, I'm still wrapping my head around the amount of testing that is needed. Because if you extrapolate that to what that means on an entire project such as WordPress, the notion of only 20% is being tested, then becomes. And I'm betting the 20% in terms of actual number of tests is already quite high, just not high enough in. [00:30:07] Speaker B: Terms of a lot of the test. Those tests are also happy path testing. So yeah, there are tests, but are the tests good enough? Yeah, I mean, that project where that 30 lines of code comes from is PHP code sniffer based project, and it's a small project, I mean, maybe 20 classes or something. I'm pretty sure that if I look at the test suite now, it's got close to or the same number as test, as WordPress core include. [00:30:45] Speaker A: Yeah, so that's a good and that's. [00:30:48] Speaker B: For a project with 20 lines of 20 classes compared to WordPress, that's a good comparison over. [00:30:56] Speaker A: We have a lot to do test base. [00:30:59] Speaker B: We need to write a lot more tests and improve the tests, and existing tests need improving, new tests need to be written and it's hard to write good tests. I'm not saying it's easy. No, definitely. [00:31:14] Speaker A: Is it by definition a senior PHP developer type thing to do? [00:31:21] Speaker B: No, everyone should write tests, I get that. [00:31:26] Speaker A: But what is it required in terms of base knowledge, and I'm guessing mostly PHP, what kind of level is required? Not just base PHP. [00:31:41] Speaker B: Like what I said, where that 30 lines of code comes from, that's a static analysis tool. Then you need to know PHP really well for something like WordPress. Now you need to know WordPress. What you do need to do is you need to understand exactly what's happening in the function, the function on the test. You need to understand that really well. You need to understand how you read code coverage, especially branch coverage, and how to generate that. You need to know the functionality being used in the function. So you might need to look up things in the PHP manual, in the WordPress manual, to know what's the underlying functionality and more than anything, to write good tests. You need to be creative, you need. [00:32:31] Speaker A: To want to break stuff, you need. [00:32:34] Speaker B: To think of scenarios, you need to be able okay, so there is a square hole, you have a round hole and you have that triangle hole. And then you know that video where people say, okay, I have a square and they put it through the square hole and then, hey, I have a triangle, and they turn it around and put it through the square hole and you have a triangle and you put it through the square hole. [00:33:03] Speaker A: So you need that sort of thinking. [00:33:05] Speaker B: Yes, you need to be able to think of lots of different data sets to give to the test to be able to try and break it. [00:33:14] Speaker A: I can sympathize with that. I'll give you a very simple example of that. So writing in Dutch, we have an expectancy of how the words are written, right? I come from a different part of the Netherlands where we have our own language, which has a different type of spelling, has different types of things that we do with our words. I was once part of the negative downside of not testing for Freesian within Dutch. It's so simple, but it's just the assumption that there was a hard assumption that word would be Dutch. It wasn't. I entered it and it crashed an entire database. Yay, simple, super simple. And I'm like, surely that can't be what I just entered. So I tried again and it crashed again. I'm like, are they expecting something else here? So I put in something else and it didn't crash. So I'm like, let me try the first thing again, and crashed again. Okay, wonderful. Somebody's not testing here. But it's indeed simple things like that. You need to be creative. You need to think of unknown scenarios. So I love giving another example of this where testing means the world. I used to work at Achmea, which is a Dutch insurance company, and I worked at HR. I was a project manager handling It. Projects, but delivering it to HR people. And what we found is that the HR people were, let's just say it very politely, not very technically inclined. So there was a great example of every single assumption that we would have in terms of surely they're not going to do that because that would be like, why would you do that? They would do that. They would click on back buttons when they're done 90% with the form just to check if oh no, wait. [00:35:20] Speaker B: The fields are filled in on the previous page, correct? [00:35:22] Speaker A: Yeah, exactly. And if you don't catch those things in terms of storing that in local memory, local storage, you're going to end up with forms that are empty every single time they do that. And there's tons of examples like that. But that job particularly told me never assume in terms of code and be as creative and make it monkey proof. Like if a monkey would hit the keyboard and stumble through whatever the interaction is. [00:35:48] Speaker B: Yeah, but if we're talking database or data and forms and HR, I mean, there are addresses without a. House number. Making a number a required field is other assumptions built into the form or on people being binary, which we all know is not the reality. [00:36:15] Speaker A: Yeah. [00:36:16] Speaker B: So a typical form will often say male, female, and hopefully nowadays they have others not to say or other specify. [00:36:30] Speaker A: Yeah. [00:36:33] Speaker B: There'S a lot of assumptions which can be built in there. And that's the whole thing with testing. If you test based on assumptions, if the assumption is, okay, this function expects a string, so I'm just going to give it string data and making sure that all the strings are handled correctly, then you've already got a massive oversized, namely what happens if a non string has been entered. [00:36:57] Speaker A: Yeah. [00:37:00] Speaker B: And that's unhappy testing and we need so many more unhappy part tests. [00:37:06] Speaker A: But is it still a fair assumption to say that it requires somebody yes. Creative of mind in terms of coming up with tests that are helping particular scenarios. [00:37:16] Speaker B: Sometimes it's also a bit nasty and wicked in mind. I mean, you can find security leaks like this as well. [00:37:23] Speaker A: Yeah. [00:37:24] Speaker B: Leaks are nearly always situations where the original coders couldn't conceive that that could possibly happen. [00:37:33] Speaker A: Yeah. [00:37:34] Speaker B: And therefore were ingested. [00:37:36] Speaker A: My favorite ones are when somebody knows the path of where the file is located, enters that in the browser, and then magically it appears or you get to do something with it. It's an obscurity fail because you think it's obscure. Nobody can see it when we're in this screen. No, but they might know it's there and they don't even have to do it manually. They can do that in a scripting way. [00:38:04] Speaker B: I have a website, one I created one of the first websites. I created a photo website. I'm not going to give you the address because it's private and it's not very exciting. I haven't really touched the site since 2005, I think. But I have an error reporting system which I built at that time. It actually sends me an email for every four or 500 status code error message, which means that I know exactly what hackers are automatically testing websites for. Yeah, I get those reported to me for that site, which is not a WordPress site. It's largely plain HTML with a little bit of sprinkling on top, but yeah, so I get all of them reported, so I know exactly what people are looking for. [00:39:05] Speaker A: It's probably something we should automate. Right. This is something that we should do by default. So I'm using Hrefs as a service to handle lots of the technical SEO side of stuff because the output, they kind of scan in the same way. Right. They will find the same errors. But just understanding that that is something you have to do in order to understand the health of your site project. [00:39:30] Speaker B: Yeah, but there's different aspects. You have the server site configuration and how that's set up and that's not always something you can automate. It test in an automated manner. No, you have the actual code and then you have the code of the WordPress core plugins, et cetera. So there's lots of different parts and lots of different levels where you need to test things and every single level can break. [00:39:57] Speaker A: Yeah, and it will. And it will. Back to my previous question, so I'm trying to think of a way to define a profile of somebody who can help with testing. Right, so it doesn't need to be a senior level PHP developer, it can be anyone. But I would assume there's a bottom layer of knowledge someone would need to have. [00:40:23] Speaker B: You need to be creative more than anything. I mean, the actual code part isn't even the most important one. I mean, the actual writing of test code is really simple. It's coming up with the data sets, which is the difficult part, with the data sets which could possibly break the code. So you do need to be able to read the actual code and the test actually understand that. If you don't understand the code and the test. And there was a project the other day, it's again an external dependency of WordPress, and I was looking together with someone else at some of that code, like how on earth can we test this? And this was really next level complicated mathematics and I was struggling to understand the code. When I'm struggling to understand the code, I'm not the right person to write the tests. [00:41:19] Speaker A: That's a good way to start the requirement list. Right, so that means a junior immediate or a senior less relevant. More relevant. Do you understand what this code is doing? Can you help then think of different scenarios of how you could possibly break its intent. [00:41:37] Speaker B: Exactly. And if there is a function which is too difficult for you to test, that's fine. There are plenty of other functions which. [00:41:47] Speaker A: Aren'T there's quite a few functions in Webers, aren't there? [00:41:50] Speaker B: Yeah. So there's lots of functionality which is at different levels. So if you want to contribute to testing, start with writing tests for the code you understand and then that will also increase your understanding of the code again, so the next time you can take on something more difficult. [00:42:09] Speaker A: Yeah, that is great advice. I'm always curious why certain elements within WordPress core development in the largest scale of the Word, why certain elements are underrepresented in attention. So performance for a very, very long time has been like a sort of stepchild type of relationship. We know it's important. Yeah, we're looking at it, but we're not really doing anything with it. So that being a dedicated team now warms my heart in many ways. Testing and coding standards and everything that comes with that sort of and I think I hear you confirming it sort of sits in the same area of we know it's important, we really do, but we're not doing enough to address all the problems that can be solved by implementing that. [00:43:17] Speaker B: The thing is, and this is in a lot of ways the same discussion a lot of development companies have with their customers. [00:43:27] Speaker A: Yeah. [00:43:29] Speaker B: Improving performance has a feasible improvement for the end user. The end user can see the site loading faster, and therefore it has a visible, tangible result for the end user. Writing tests does not have a feasible result for the end user. What writing test does is making sure there are less bugs in the code. Making sure that any bugs, which we may not have discovered because they weren't previously bugs, because PHP has changed, will not reach the end user. I will say user doesn't see any result other than that maybe they have less bugs to report. Except the problem with that is having less bugs to report is not something tangible for an end user. And the general end user won't report bugs anyway because they wouldn't even know where to start to describe the bug. [00:44:29] Speaker A: Yeah. Or even to find where to do it, describing the thing. [00:44:36] Speaker B: It's not a tangible improvement for the end user. Other than that, nothing has changed. It all looks the same, it just underneath dirt. It works better and it continues to work. [00:44:49] Speaker A: So what do we need to do to fix this then? How do we attract the right people? How do we lower the bar for entry? [00:44:59] Speaker B: I think we need to actually raise the bar for entry, and not so much raise the bar for entry to the project at all, but raise the bar for entry of code to the code base. In my opinion, we should. [00:45:11] Speaker A: Okay, so reverse it. [00:45:12] Speaker B: Essentially make it a requirement for any patch to go into WordPress functionality to have test. That means that any code which is touched as of this moment would start to be tested. And that means both existing code as well as new code. So existing code, at least for departure, changing, needs to be tested. And once you write a test for something, it's easier to expand that test, cover larger parts of the function. I can write a test if there's a function of typical WordPress thousand lines. I can write a test to just hit those three lines I'm changing. But once that test is in place, and if I set it up with a data provider, it's going to be so much easier for other people to add more test cases to it. They don't even have to write the test code anymore that's already there. They just need to add test cases. [00:46:11] Speaker A: So that's essentially where we need to start. [00:46:14] Speaker B: Yeah. Raise the bar for allowing code to be committed to core. [00:46:19] Speaker A: Yeah. That's an interesting I don't think I've ever heard that one before. [00:46:23] Speaker B: I've been saying this for quite a while. [00:46:26] Speaker A: Yeah. And it makes all the sense. It's hard to start with from scratch to something, but once something is there, it's a lot easier to expand on it. I would say for the most part, that's how my mind works in terms of I don't particularly enjoy writing code from scratch. In fact, I think I've stopped doing it for six, seven years, maybe even longer. It just does not give me joy or doesn't spark me. Joy, I think is the right way. But if something is existing, let's just say the architecture is incorrect, it's done in the wrong way or the wrong sequence, it's much easier for me to have joy in and say, hey, I think we can optimize here doing that and changing things around. What do you think? That is a much more interesting conversation. [00:47:22] Speaker B: So imagine if you it's all the more important to have tests in place. [00:47:26] Speaker A: To validate that you know exactly where I was going. So the next thing then is if you look at from that perspective, we have tests, something is changed. Wait, is this a smart thing to do? Because now we have to in the test show and can we improve this? I go back to the performance team. We're now doing performance tests, right? And we do them even between release candidates to see if there's regression or not. And wouldn't you know it, in the last two versions, six two and six one, we've had regressions on performance. This is telling me that in previous times we've had those as well, but we didn't test for it. So they're still there. Yeah. That's crazy. [00:48:13] Speaker B: Not unsurprising. [00:48:15] Speaker A: Not unsurprising, but it's crazy to consider if you're not testing for it, you won't find it. [00:48:22] Speaker B: Tests can't fill if you don't have them. [00:48:24] Speaker A: Yeah. [00:48:31] Speaker B: This is the typical mantra. The easiest life for a developer is not having tests because then nothing drags ever. Yeah, reality is different. [00:48:40] Speaker A: Yeah. But the mindset is certainly as such. [00:48:43] Speaker B: And that's why, especially for projects I inherit, like Request, I mean, I in a way inherited it because it was no longer maintained. I'm not intimately familiar with everything it does. I honestly don't ask me anything about requests because I probably don't have the answer, which is all of a reason why I'm writing more and more tests for it. Because otherwise I cannot merge things pulled by other people with confidence because I don't know code base well enough myself to sure it doesn't break anything. [00:49:17] Speaker A: Was that project started by Ryan McHugh? [00:49:24] Speaker B: Life happens, people have no blame there. [00:49:28] Speaker A: Just curious talking about the same thing. [00:49:30] Speaker B: But yeah, when for me taking over, it's like, yeah, this was actually a code base which did have tests, which it didn't have low code coverage. It actually did have tests, but it needs a lot more tests and a lot more unhappy about testing. The tests needed improving. That's one of the main contributions I've been doing to Request is getting that up to scratch. [00:49:57] Speaker A: Yeah. Well, to help us future on behalf of the entire WordPress project, I'd like to thank you for what you're doing, and I also like to thank you for being on this podcast. And I have one last question, if I may. [00:50:17] Speaker B: You may. [00:50:19] Speaker A: I would love for you to come here because we live in the same country and do a live podcast. So there's a few more things I think I'd love to talk to you about. So I'd like to invite you for a second one, if that's something you would like. [00:50:35] Speaker B: Very happy to. And I can actually think of a topic. [00:50:39] Speaker A: Awesome. Well, don't share it yet. Surprise me. [00:50:43] Speaker B: Sure. [00:50:44] Speaker A: Unless you think, like, a proper cliffhanger. [00:50:48] Speaker B: Let's get people excited. [00:50:50] Speaker A: Okay, go. [00:50:52] Speaker B: We're still very much hoping to release WordPress as coding standard 3.0 soon ish. [00:50:58] Speaker A: Yeah. [00:50:59] Speaker B: And I think it would be a lot has changed, and that's a whole different story. But I think it would be really good to have maybe a session on WordPress coding standards and why they're important and how they can help you and why 3.0 took such a long time to get released. [00:51:21] Speaker A: I'd love that. I would love that. Okay, let's settle that then. But thank you for today, and if you want to know more about Juliet, where can you find information? [00:51:34] Speaker B: I'm on Twitter and I'm on Mustadom. Both with the handle JRF underscore NL. So my initials, Juliet Ryan schwarmer at. And I'm on the WordPress coursela. And if it's anything to do with code, please use track or GitHub. And don't DM me. [00:51:56] Speaker A: Yes. Thank you so much. [00:52:00] Speaker B: Thank you for having me. It was a pleasure talking with you. Likewise.

Other Episodes

Episode 33

May 03, 2024 00:52:25
Episode Cover

Inside WordPress Security: Conversations with security veteran Tom Raef

This comprehensive conversation delves into the world of WordPress security through the lens of Tom Raef, a seasoned security expert with a history dating...

Listen

Episode 13

September 15, 2023 01:01:10
Episode Cover

Ian Svoboda: On Block Themes, Custom Block development and a whole lotta WP CLI

In this podcast I talk with Ian Svoboda about themes and Custom Blocks. He covers how to develop themes and Custom Blocks using the...

Listen

Episode 3

June 02, 2023 00:45:26
Episode Cover

Solving Human Resources for WordPress Companies with James Giroux

James Giroux sat down with me for an episode of the Within WordPress podcast. We talked about how his company Team WP is solving...

Listen