RSS feed [root] /software_engineering /weblog /testing



title search:


Thu Nov 09 11:20:56 GMT 2023


(google search) (amazon search)
download zip of files only

Sun Nov 05 12:32:47 GMT 2023 From /weblog/software_engineering/testing


Getting The Most Test Coverage With The Least Test Code - , Interesting question to ask, but I don't buy the arguement personally, because Most Test Coverage is not related to Most Effective Test in most case

mutation-testing, injecting wrong code to provide unit test is working -

The Great Code Coverage Holy Wars of the 21st Century -[..]ly-wars-of-the-21st-century-6fb11e7acce4

(google search) (amazon search)

Fri Sep 01 01:27:49 GMT 2023 From /weblog/software_engineering/testing

case study

How google doing test -[..]/how-google-tests-software-part-two.html

How facebook doing test? -[..]nd-of-automated-testing-does-Facebook-do

How to test map-reduce application -[..]ting-mapreduce-with-the-adapter-pattern/

The Knightmare Explained - ,

Example of using mocking tools -

How sqlite tested -[..]014/06/how-do-i-make-testing-faster.html[..]3/29/test-driven-thermostat-development/

Story of testing with in-memory DB -[..]ated-integration-tests-failure-scenarios[..]ase-do-we-have-to-do-regression-testing/

What is an A/B Test? -

The complexity behind eradicating test flakiness -[..]arks-chew-on-network-cables-c0cf8c8b015b

(google search) (amazon search)

Wed Dec 22 13:25:19 GMT 2021 From /weblog/software_engineering/testing


There is a lot of discussion about this blog -

Which talk about classification of functional test and unit test. Nice to read. You can also refer to the following article:

Checking vs. testing -

Focusing on testing -[..]012/03/18/introducing-depth-of-test-dot/

Verification and Validation in Testing -[..]/verification-and-validation-in-testing/

(google search) (amazon search)

Thu Dec 09 23:31:12 GMT 2021 From /weblog/software_engineering/testing

best practices

Consider the risk of not being tested -[..]sting-on-toilet-risk-driven-testing.html

Fidelity, Resilience, Precision -[..]testing-on-toilet-effective-testing.html

Attributes that unit test should have: Functionality, Accuracy, Instant, Locator -

Isolation -[..]/2012/04/is-your-unit-test-isolated.html

Tips of keeping unit tests running fast -

Testing Patterns -

Continuously to break thing so that we know our system is solid -[..]11/04/working-with-the-chaos-monkey.html

First rule -[..]08/10/01/nothing_is_too_trivial_to_test/

Test first/last is not important? Unit test either? What do you think?

* The name of the test should describe the requirement of the code
* There should be at least one test for each requirement of the code. Each possible path through of the code is a different requirement#
* Test the goal of the code, not the implementation[..]AppQuality&asrc=EM_NLN_761453&uid=703565[..]og/2007/08/how-not-to-run-beta-test.html

The teaser: Fast, Isolated, Repeatable, Self-validating, and Timely. -[..]es/2007/08/02/not-a-task-but-an-approach

Corner cases -[..]s/2007/02/testheuristicscheatsheetv1.pdf

One of the targets of TDD coding -[..]e-code-is-about-managing-complexity.aspx

Design for unit test -[..]es/content/DesigntoUnitTest/article.html

Push and Pull approach -

When not to test -

Test the story, rather than the implementation -[..]ving-to-scenario-based-unit-testing.html

Test the configuration -[..]TSS10ctqa&asrc=EM_NLN_8746433&uid=703565

Feel the deep synergy of design and test constraint -[..]athers_blog/2007/09/the-deep-synerg.html

Another set of principles for automated testing -[..]of-principles-for-automated-testing.html

Priority for tester -[..]

A lot of links -[..]-links-biased-toward-exploratory-testing[..]nit-tests-5-principles-for-unit-testing/

Some information and suggestion about setting up a local integrated testing environment -[..]

Virtual Panel: Code-to-Test Ratios, TDD and BDD -

A test is complete when its body contains all of the information you need to understand it, and concise when it doesn't contain any other distracting information. -[..]ting-on-toilet-what-makes-good-test.html

Discussion about naming the tests -[..]14/03/17/getting-junit-test-names-right/[..]ting/writing-clean-tests-naming-matters/[..]esting-on-toilet-test-behaviors-not.html <- Test behaviour, not method. This will make your tests more resilient since adding new behaviors is unlikely to break the existing tests, and clearer since each test contains code to exercise only one behavior.

Tips on having better assertion or cleaner test -[..]ur-test-code-with-custom-assertions.html[..]-friday-most-internal-dsls-are-outdated/[..]ertions-with-a-domain-specific-language/

Good to prevent setup and tearDown?

#1 Treat Test Code as Production Code
#2 Use Test Patterns to achieve great readability
#3 Avoid Unreliable Tests
#4 Test at The Appropriate Level
#5 Do Use Test Doubles[..]insights/blog/write-better-tests-5-steps[..]om/core-java/junit/junit-best-practices/

Document ‘Why’, specify ‘What’, automate ‘How’ -[..]/agile/2016/05/24/large-test-suites.html

Good typing to prevent un-necessary test -[..]com/2014/12/09/typed-language-tdd-part1/

Unit Tests Are FIRST: Fast, Isolated, Repeatable, Self-Verifying, and Timely -[..]e-self-verifying-and-timely-a83e8070698e

(google search) (amazon search)

Fri Jul 09 06:03:29 GMT 2021 From /weblog/software_engineering/testing


If you write the test after you've written the code, it's much more likely that you'll write the tests that will pass what you've written.

That's just how our brains work.

If you determine the criteria for whether a decision is good after you've already made the decision, it's much more likely that you'll create criteria that justifies the decision that was just made.

That's just how our brains work.

Determine how to assess whether something is good before you implement it and/or before you make a decision. Otherwise, you will tend to be emotionally attached to what you just did, what you just decided.

My Top 5 ways to reproduce a "Hard to Reproduce" Bug! -[..]/my-top-5-ways-to-reproduce-hard-to.html

Common TDD issue and suggested solution -[..]utomated-test-problems-address-root.html[..]om/biblio/StudiesOfTestDrivenDevelopment[..]ptions-with-test-driven-development.html[..]014/05/told-you-that-testing-is-a-waste/[..]-best-thing-has-happened-software-design[..]evelopment-is-dumb-fight-me-a38b3033280c

Does this mean I think you should skip TDD for programs you’re going to run once and then throw them away?

Well, I’m closer to OK with that than I am to the other cases here, but I often spend enough time editing and rerunning to make me think there was probably a central bit of the program that would have benefited from some TDD.

What TDD is -- and isn't -- like. -

(google search) (amazon search)

Fri Sep 04 05:26:55 GMT 2020 From /weblog/software_engineering/testing


A nice example of convert dbunit from using junit to testng -[..]h/blog.php?name=philzoio&mydate=20050826

redirect junit output to a file -

invokeNCopiesWithTimeout, run a test ( in runnable() ) for n times and see if it ok for multi-thread likely or not -[..]/running-n-foreground-tasks-in-java.html

Use theories to execute test with a lot difference input, probably possible for all cases -[..]010/01/31/new-feature-of-junit-theories/[..]using-junit-4-theories-to-test-contracts

The other features, rule, perform some rule on test cases -

Adding timeout -

Testing exceptions at Junit -[..]ting-expected-exceptions-with-junit.html[..]ays-of-handling-exceptions-in-junit.html[..]-java-unit-testing-frameworks-tools.html

Skip test for some cases, or create test only for some cases -[..]unit-rule-to-conditionally-ignore-tests/

What new in junit5 -[..]it-5-next-generation-testing-on-the-jvm/

Compare with jupiter -[..]8-09-13-junit-4-core-vs-jupiter-api.html

Beyond the simple: An in-depth look at JUnit 5’s nested tests, dynamic tests, parameterized tests, and extensions -[..]tests-parameterized-tests-and-extensions

(google search) (amazon search)

Sun Jan 05 12:05:34 GMT 2020 From /weblog/software_engineering/testing


WatiN, alternative of selenium -[..]2/how-to-unit-test-watin-mbunit-and.html another person saying this is better -

And someone work hard on make FITness and Selenium work together, more information can be found at[..]m/2006/09/fitnesse-selenium-wrapper.html[..]ng-web-tests-with-fitnesse-and-selenium/

A story of how to overcome various issues of applying selenium in acceptance testing -[..]07/08/selenium_is_the_pain_worth_it.html[..]9/webtest-vs-selenium-webtest-wins-13-5/ Why the functional test is important

How many resources should put in a functional test?

Kent discuss about how to balance low level unit test and high-level functional test -

(google search) (amazon search)

Thu Aug 22 00:00:00 GMT 2019 From /weblog/software_engineering/testing


The evil test:

1. Evil tests create a lock on how the code is implemented.
2. Cause duplication.
3. Builds uncertainty on the tests (red is meaningless).
4. Decrease productivity.
5. Discourage change.

use thread in junit -[..]blog/2007/11/05/false-positives-in-junit

Don't try to test everything -[..]ay?entry=unit_testing_avoiding_extremism

Why TDD fail? Because test is too complicate to write - ( I agree it a lot )

Hard to test something? Unreadable tests? Slow running tests? It takes too long to write a test? Some solution suggested -[..]/last-d-in-tdd-means-more-than-just.html

Comment out test so that the code compile -

A list of TDD antipattern -

And the long discussion using random in unittest -[..]rivendevelopment/message/20458?var=1&l=1
Here is an example of using random in unittest, it actually same for every new instance! -

Test abstraction smells -[..]com/2011/11/test-abstraction-smells.html

Basically, we should keep it simple, and driven the development rather than post test -[..]stteam/Posts/13-01-24/top-5-tdd-mistakes

A good list of anti-pattern -[..]2014/05/06/getting-unit-testing-to-fail/
Test rely on shell script return is difficult to maintain, say the script running at background can cause problem -[..]m/2014/01/10/another-shell-test-pitfall/

Six Things That Go Wrong With Discussions About Testing -

(google search) (amazon search)

Mon Apr 29 02:54:01 GMT 2019 From /weblog/software_engineering/testing


Customer attitude will affect how the software engineering workflow a lot:[..]gprog/message/7420?threaded=1&var=1&p=24

Do you think this piece of code are too simple to have test case to cover? read this -[..]t-the-software-craftsmanship-conference/

Related blog, finding the common between taking photo and software testing -[..]hotographic-testing-lessons-learned.html . However, there is one big difference, it is harder to define a good photo than bug feel software

If it is good to put developer as tester? -[..].com/2011/03/programmers-as-testers.html

In fact I am kind of agree about this article, but maybe I just not TDD hard enough -[..]ectural-meltdown-around-iteration-three/

Tooling, Insight and Humility -[..]reasons-for-testers-to-learn-to-program/

Insufficient Requirements is not an excuse -[..]-blog-posts-9-insufficient-requirements/

What is the meaning of working -[..]14/03/very-short-blog-posts-14-it-works/

How Much is Enough? Testing as Story-Telling -

(google search) (amazon search)

Tue Apr 09 10:38:55 GMT 2019 From /weblog/software_engineering/testing


Yet other discussion of Stubs, Fakes, and Mock Objects -

While reviewing this article, Marty Andrews pointed out an important insight: there are two ways to use a mock object, either as a testing technique or as a design technique, and the intended use helps shape the available choices.

Someone post some arguement that using mock probably a bad idea , which look like a endless arguement... -[..]xRuiz?entry=mocks_for_concrete_class_can

Alberto Savoia on Testing the Untestable, raise a point that mocking make testing more portable, it probably true, but database setup may be not that difficult to port. -[..]ut-isolation-but-about-responsibilities/[..]ing-the-invasion-of-dots-and-parentheses

Fakes should have their own tests -[..]/testing-on-toilet-fake-your-way-to.html

Again, other post on pros and cons for difference approach -

(google search) (amazon search)

Thu Mar 30 06:47:41 GMT 2017 From /weblog/software_engineering/testing

test data

Suggestion of how to manage the test datas -[..]citcon-melbourne-2008-managing-test.html

Difference Pattern of managing test datas -[..]damentals-of-data-testing-setup-and.html

How to get the data feed and design automated test trading system -[..]m/2011/02/16/trading-system-testability/

False assumption about time -[..]alsehoods-programmers-believe-about-time

Create your own clock -[..]3/12/the-virtual-clock-test-pattern.html

Just change the return of Calendar -

Discussion about creating test data -[..]est-data-builders-and-object-mother.html[..]r-data-access-code-unit-tests-are-waste/

Dummy, fake, stub, spy and mock -[..]

Generation of test data -[..]/tools/open-source-test-data-generators/

Using test container, and compare it with other test data solution -[..]28/database-testing-with-testcontainers/

(google search) (amazon search)

Tue Nov 15 08:56:18 GMT 2016 From /weblog/software_engineering/testing


Being proactive! -[..]to-be-first-class-citizen-as-tester.html

Well, of course it say yes...... -[..]6/27/software-testers-need-learn-coding/

(google search) (amazon search)

Thu Feb 18 03:12:10 GMT 2016 From /weblog/software_engineering/testing


UnitTest DB, check if proper index applied -[..]og/2012/09/test-driving-database-indexes[..]an-integration-testing-with-junit-rules/[..]ust-say-no-to-more-end-to-end-tests.html

(google search) (amazon search)

Thu Apr 09 04:10:12 GMT 2015 From /weblog/software_engineering/testing

GUI test

How to write unit test for RCP application in eclipse -

and functional robot tester -[..]ame/2007/12/18/announcing-swtbot-01.html

A lot of links of tools -

CI for eclipse RCP -

Don’t write E2E tests instead of UI tests. Instead write unit tests and integration tests beside the UI tests.
Hermetic tests are the way to go.
Use dependency injection while designing your app.
Build your application into small libraries/modules, and test each one in isolation. You can then have a few integration tests to verify integration between components is correct .
Componentized UI tests have proven to be much faster than E2E and 99%+ stable. Fast and stable tests have proven to drastically improve developer productivity.

(google search) (amazon search)

Mon Jul 14 08:25:03 GMT 2014 From /weblog/software_engineering/testing


Automated deployment test on VM -[..]tion-using-docker-maven-and-jenkins.html

(google search) (amazon search)

Thu Jul 10 02:38:17 GMT 2014 From /weblog/software_engineering/testing


Create a thread-pool to execution concurrency -[..]how-to-find-a-concurrency-bug-with-java/[..]com/2009/10/14/unit-testing-concurrency/[..]sting-multi-threaded-code-with-easymock/

Discussion about a tool that google used to test data racing -[..]adsanitizer-slaughtering-data-races.html

(google search) (amazon search)

Fri Apr 25 01:24:17 GMT 2014 From /weblog/software_engineering/testing


Experience of using load testing tool - , in his opinions, look like is the best[..]eparing-for-load-testing-best-practices/

(google search) (amazon search)

Wed Dec 04 05:03:38 GMT 2013 From /weblog/software_engineering/testing


Various discussion about debugger

Debuggers are a wasteful Timesink -
The Power of Debuggers -

And a lot of links -

I feel the same, most of the time rely on debugger is not that nice -

Here are the types of debuggers we cover:
CLI debuggers
IDE debuggers
Build you own debugger
Heap dumps
Historical debuggers
Dynamic tracing
Production debugging

(google search) (amazon search)

Tue Feb 05 13:14:22 GMT 2013 From /weblog/software_engineering/testing

nice bug!

computer system have to be use it very careful? or it should be have the user to prevent mistake?

hard to reproduce bug -[..]1_h&elq=2a3777bae66d4ddfb7650a070c8ca30b

(google search) (amazon search)

Mon Oct 01 09:14:51 GMT 2012 From /weblog/software_engineering/testing


Use Scala closure to make test look nicer -

Specification base testing -

(google search) (amazon search)

Thu Nov 03 17:41:15 GMT 2011 From /weblog/software_engineering/testing

exploring test

Using mindmap as testing tool -[..]02/28/using-mind-maps-for-test-planning/

Chat about exploring test -[..]g-or-scripted-testing-which-comes-first/

How to effective use notebook with exploratory-testing -[..]4/exploratory-testing-recording-and.html

(google search) (amazon search)

Mon Oct 03 16:22:03 GMT 2011 From /weblog/software_engineering/testing


By far the biggest reason I have seen for the performance discrepancy above is not due to a faulty test but due to the stress test being executed on wildly different data sets than what is in production. -[..]on-flaw-in-software-performance-testing/[..]0-open-source-performance-testing-tools/

Discuss about performance management, a process to monitor and control performance details -[..]1/09/15/why-do-you-do-apm-in-production/

(google search) (amazon search)

Mon Jun 20 16:16:16 GMT 2011 From /weblog/software_engineering/testing

Release Checklist

Various discussion about the checklist: . But I think practice CI is more important:

Joey discuss on beta testing... I feel he requirement is too high for small team -

More discussion - and a checklist for SCRUM

Integraton test concern -[..]integration-points-4-main-considerations[..]

(google search) (amazon search)

Wed Apr 27 16:36:26 GMT 2011 From /weblog/software_engineering/testing

refactoring unit test

Idea of how to refactoring test cases, summary :

Refactor production code with the tests passing. This helps you determine that the production code still does what it is meant to.
Refactor test code with the tests failing. This helps you determine that the test code still does what it is meant to.[..]07/04/tott-refactoring-tests-in-red.html

Collections of idea of testing private methods - , may be it is a signal of refactoring?

The other way of testing private , via compare with difference implementation -[..]1/unit-testing-private-methods-like.html

Explain the problem of non-Determinism in testing, common reason of why tests becoming non-Deterministic, and common solution -

(google search) (amazon search)