An issue is raised when a test method uses documentation comment markers (like comment-based tags) instead of proper metadata attributes or decorators that the testing framework requires to identify and execute tests.

In Java TestNG specifically, this means using Javadoc-style tags like @test or @beforeMethod in comments instead of the proper Java annotations @Test and @BeforeMethod.

Why is this an issue?

Testing frameworks initially supported documentation comment tags for configuring test methods. For example, developers could write special markers in documentation comments to designate methods as tests. While this approach worked in older framework versions, it has significant drawbacks compared to modern declarative configuration mechanisms.

Documentation comment tags are simply strings embedded in comments. The compiler doesn’t check them, and development environments cannot validate them during development. This means typos or incorrect usage won’t be caught until runtime, making bugs harder to find.

Modern testing frameworks use first-class language constructs that declare test configuration directly in code. These are part of the language itself, so the compiler validates them at compile time. This catches errors early and makes the code more reliable.

Language-native declarations also integrate better with development tools. IDEs can provide autocomplete, navigation, and refactoring support. When you rename a test class or reorganize code, these declarations are automatically updated, while comment-based strings remain unchanged.

Using documentation comment tags in new code creates technical debt. The code appears outdated and may confuse developers who are familiar with standard framework practices. It also makes the codebase inconsistent if some tests use language-native declarations while others use comment tags.

In Java, TestNG uses annotations like @Test, @BeforeMethod, and @AfterMethod instead of the older Javadoc-style tags such as /** @test */.

What is the potential impact?

Using documentation comments for metadata instead of structured metadata mechanisms reduces code maintainability and increases the risk of errors:

• Delayed error detection: Typos or configuration mistakes in comment-based metadata are only discovered at runtime, potentially during test execution in production environments.
• Harder refactoring: When restructuring code, development tools cannot automatically update string values embedded in comments, leading to broken test configurations.
• Reduced discoverability: New team members may not recognize comment-based test configuration, making the codebase harder to understand.
• Limited tooling support: Build tools and test runners may have reduced functionality with comment-based configuration compared to structured metadata mechanisms.

How to fix it

Replace Javadoc tags with the corresponding TestNG annotations. The most common conversions are:

• /** @test */ becomes @Test
• /** @beforeMethod */ becomes @BeforeMethod
• /** @afterMethod */ becomes @AfterMethod
• /** @beforeClass */ becomes @BeforeClass
• /** @afterClass */ becomes @AfterClass

Import the annotation classes from org.testng.annotations package. Remove the Javadoc comment or keep it for documentation purposes without the tag.

Code examples

Noncompliant code example

/**
 * @test
 */
public void shouldValidateInput() { // Noncompliant
    // test code
}

Compliant solution

import org.testng.annotations.Test;

@Test
public void shouldValidateInput() {
    // test code
}

Resources

Documentation

• TestNG Documentation - Annotations
• TestNG Documentation - TestNG Test Methods