--- name: Negative Test Generator description: Systematically generate negative test cases covering invalid inputs, unauthorized actions, missing required fields, exceeded limits, and malformed request payloads version: 1.0.0 author: Pramod license: MIT tags: [negative-testing, error-handling, invalid-input, boundary-testing, validation-testing, robustness, fault-tolerance] testingTypes: [unit, integration, security] frameworks: [jest, vitest, pytest] languages: [typescript, javascript, python, java] domains: [web, api] agents: [claude-code, cursor, github-copilot, windsurf, codex, aider, continue, cline, zed, bolt, gemini-cli, amp] --- # Negative Test Generator Skill You are an expert QA engineer specializing in negative testing and robustness verification. When the user asks you to create, review, or improve negative test cases, follow these detailed instructions to systematically generate tests that verify the system correctly rejects invalid inputs, handles error conditions gracefully, and maintains data integrity under adversarial conditions. ## Core Principles 1. **Every input has an invalid twin** -- For every valid input a system accepts, there exists a family of invalid inputs that must be rejected. Negative testing maps this family systematically, not randomly. 2. **Error messages are features** -- A good error message tells the user what went wrong, where, and how to fix it. Negative tests must verify not just that errors occur, but that the error response is actionable and accurate. 3. **Validation boundaries are contract boundaries** -- The boundary between valid and invalid input is the most defect-dense region of any system. Test one step inside and one step outside every boundary. 4. **Fail safely, never silently** -- A system that silently accepts invalid input is more dangerous than one that crashes. Negative tests verify that rejection is explicit, logged, and does not corrupt state. 5. **Type violations are the first line of defense** -- Before testing business logic validation, test type-level violations. Sending a string where a number is expected should produce a type error, not a business logic error. 6. **Absence is a value** -- null, undefined, empty string, missing field, and empty array are five distinct concepts. Each must be tested independently because systems handle them differently. 7. **Composition multiplies invalid states** -- If a form has 5 fields and each has 4 invalid variants, the negative test space is not 20 but potentially exponential. Use pairwise testing to manage combinatorial explosion. 8. **Security testing starts with negative testing** -- SQL injection, XSS, and path traversal are negative test cases with security implications. Every input field is a potential attack vector. 9. **Concurrent invalid operations reveal race conditions** -- Sending two conflicting requests simultaneously is a negative test that most developers never write but production always executes. 10. **Error handling must not leak internals** -- Stack traces, database names, file paths, and internal IDs in error responses are negative test findings with security implications. ## Project Structure ``` tests/ negative/ generators/ input-type-violations.ts missing-fields.ts boundary-violations.ts format-violations.ts injection-payloads.ts concurrent-conflicts.ts helpers/ error-assertions.ts payload-builder.ts type-fuzzer.ts tests/ api/ create-user.negative.test.ts update-order.negative.test.ts authentication.negative.test.ts authorization.negative.test.ts validation/ email-validation.negative.test.ts number-validation.negative.test.ts date-validation.negative.test.ts string-validation.negative.test.ts payloads/ malformed-json.negative.test.ts oversized-payload.negative.test.ts content-type-mismatch.negative.test.ts concurrency/ race-condition.negative.test.ts duplicate-submission.negative.test.ts config/ negative-test.config.ts data/ injection-strings.json boundary-values.json ``` ## Input Type Violation Generator The foundation of negative testing is systematically generating invalid type variants for every field. ```typescript // input-type-violations.ts type FieldType = 'string' | 'number' | 'boolean' | 'email' | 'url' | 'date' | 'uuid' | 'enum' | 'array' | 'object'; interface FieldDefinition { name: string; type: FieldType; required: boolean; minLength?: number; maxLength?: number; min?: number; max?: number; pattern?: string; enumValues?: string[]; format?: string; } interface NegativeTestCase { name: string; field: string; category: NegativeCategory; input: any; expectedError: { status?: number; code?: string; messagePattern?: RegExp; field?: string; }; } type NegativeCategory = | 'type_violation' | 'missing_required' | 'boundary_violation' | 'format_violation' | 'injection' | 'null_undefined' | 'empty_value' | 'overflow' | 'encoding' | 'special_characters'; function generateTypeViolations(field: FieldDefinition): NegativeTestCase[] { const cases: NegativeTestCase[] = []; const typeInvalids: Record = { string: [123, true, [], {}, 0, -1, NaN, Infinity], number: ['abc', '', true, [], {}, 'NaN', '123abc', null], boolean: ['yes', 'no', 1, 0, 'true', 'false', '', null], email: [123, true, [], {}, null], url: [123, true, [], {}, null], date: [123, true, [], {}, 'not-a-date', null], uuid: [123, true, [], {}, 'not-a-uuid', null], enum: [123, true, [], {}, null], array: ['string', 123, true, {}, null], object: ['string', 123, true, [], null], }; const invalids = typeInvalids[field.type] || []; for (const invalidValue of invalids) { cases.push({ name: `${field.name}: should reject ${typeof invalidValue} (${JSON.stringify(invalidValue)}) when ${field.type} expected`, field: field.name, category: 'type_violation', input: invalidValue, expectedError: { status: 400, code: 'VALIDATION_ERROR', messagePattern: new RegExp(`${field.name}.*(?:invalid|expected|must be)`, 'i'), field: field.name, }, }); } return cases; } function generateMissingFieldCases(fields: FieldDefinition[]): NegativeTestCase[] { const cases: NegativeTestCase[] = []; for (const field of fields.filter(f => f.required)) { // Missing entirely cases.push({ name: `${field.name}: should reject when required field is missing entirely`, field: field.name, category: 'missing_required', input: undefined, expectedError: { status: 400, code: 'VALIDATION_ERROR', messagePattern: new RegExp(`${field.name}.*required`, 'i'), field: field.name, }, }); // Explicitly null cases.push({ name: `${field.name}: should reject null for required field`, field: field.name, category: 'null_undefined', input: null, expectedError: { status: 400, code: 'VALIDATION_ERROR', field: field.name, }, }); // Explicitly undefined cases.push({ name: `${field.name}: should reject undefined for required field`, field: field.name, category: 'null_undefined', input: undefined, expectedError: { status: 400, code: 'VALIDATION_ERROR', field: field.name, }, }); // Empty string (for string types) if (field.type === 'string' || field.type === 'email' || field.type === 'url') { cases.push({ name: `${field.name}: should reject empty string for required field`, field: field.name, category: 'empty_value', input: '', expectedError: { status: 400, code: 'VALIDATION_ERROR', field: field.name, }, }); cases.push({ name: `${field.name}: should reject whitespace-only string for required field`, field: field.name, category: 'empty_value', input: ' ', expectedError: { status: 400, code: 'VALIDATION_ERROR', field: field.name, }, }); } // Empty array (for array types) if (field.type === 'array') { cases.push({ name: `${field.name}: should reject empty array for required field`, field: field.name, category: 'empty_value', input: [], expectedError: { status: 400, code: 'VALIDATION_ERROR', field: field.name, }, }); } } return cases; } ``` ## Boundary Violation Generator ```typescript // boundary-violations.ts function generateBoundaryViolations(field: FieldDefinition): NegativeTestCase[] { const cases: NegativeTestCase[] = []; // String length boundaries if (field.type === 'string' || field.type === 'email' || field.type === 'url') { if (field.minLength !== undefined) { cases.push({ name: `${field.name}: should reject string shorter than minLength (${field.minLength})`, field: field.name, category: 'boundary_violation', input: 'a'.repeat(Math.max(0, field.minLength - 1)), expectedError: { status: 400, code: 'VALIDATION_ERROR', messagePattern: new RegExp(`${field.name}.*(?:min|short|least|minimum)`, 'i'), }, }); } if (field.maxLength !== undefined) { cases.push({ name: `${field.name}: should reject string exceeding maxLength (${field.maxLength})`, field: field.name, category: 'boundary_violation', input: 'a'.repeat(field.maxLength + 1), expectedError: { status: 400, code: 'VALIDATION_ERROR', messagePattern: new RegExp(`${field.name}.*(?:max|long|exceed|maximum)`, 'i'), }, }); // Significantly exceeding max length cases.push({ name: `${field.name}: should reject extremely long string (10x maxLength)`, field: field.name, category: 'overflow', input: 'a'.repeat(field.maxLength * 10), expectedError: { status: 400, code: 'VALIDATION_ERROR', }, }); } } // Numeric boundaries if (field.type === 'number') { if (field.min !== undefined) { cases.push({ name: `${field.name}: should reject number below minimum (${field.min})`, field: field.name, category: 'boundary_violation', input: field.min - 1, expectedError: { status: 400, code: 'VALIDATION_ERROR', messagePattern: new RegExp(`${field.name}.*(?:min|least|greater)`, 'i'), }, }); cases.push({ name: `${field.name}: should reject large negative number`, field: field.name, category: 'boundary_violation', input: -Number.MAX_SAFE_INTEGER, expectedError: { status: 400, code: 'VALIDATION_ERROR' }, }); } if (field.max !== undefined) { cases.push({ name: `${field.name}: should reject number above maximum (${field.max})`, field: field.name, category: 'boundary_violation', input: field.max + 1, expectedError: { status: 400, code: 'VALIDATION_ERROR', messagePattern: new RegExp(`${field.name}.*(?:max|exceed|less)`, 'i'), }, }); } // Special numeric values cases.push({ name: `${field.name}: should reject NaN`, field: field.name, category: 'type_violation', input: NaN, expectedError: { status: 400, code: 'VALIDATION_ERROR' }, }); cases.push({ name: `${field.name}: should reject Infinity`, field: field.name, category: 'type_violation', input: Infinity, expectedError: { status: 400, code: 'VALIDATION_ERROR' }, }); cases.push({ name: `${field.name}: should reject -Infinity`, field: field.name, category: 'type_violation', input: -Infinity, expectedError: { status: 400, code: 'VALIDATION_ERROR' }, }); cases.push({ name: `${field.name}: should handle -0 correctly`, field: field.name, category: 'type_violation', input: -0, expectedError: { status: 400, code: 'VALIDATION_ERROR' }, }); } return cases; } ``` ## Format Violation Generator ```typescript // format-violations.ts function generateFormatViolations(field: FieldDefinition): NegativeTestCase[] { const cases: NegativeTestCase[] = []; if (field.type === 'email') { const invalidEmails = [ { value: 'notanemail', reason: 'missing @ symbol' }, { value: '@domain.com', reason: 'missing local part' }, { value: 'user@', reason: 'missing domain' }, { value: 'user@.com', reason: 'domain starts with dot' }, { value: 'user@domain', reason: 'missing TLD' }, { value: 'user @domain.com', reason: 'contains space' }, { value: 'user@dom ain.com', reason: 'space in domain' }, { value: 'user@@domain.com', reason: 'double @ symbol' }, { value: 'user@domain..com', reason: 'consecutive dots in domain' }, { value: '', reason: 'data URI with script' }, ]; for (const { value, reason } of invalidUrls) { cases.push({ name: `${field.name}: should reject invalid URL -- ${reason}`, field: field.name, category: 'format_violation', input: value, expectedError: { status: 400, code: 'VALIDATION_ERROR', messagePattern: /url.*invalid|invalid.*url/i, }, }); } } if (field.type === 'date') { const invalidDates = [ { value: '2024-13-01', reason: 'month 13' }, { value: '2024-02-30', reason: 'February 30th' }, { value: '2024-00-01', reason: 'month 0' }, { value: '2024-01-32', reason: 'day 32' }, { value: '2024-01-00', reason: 'day 0' }, { value: '24-01-15', reason: 'two-digit year' }, { value: 'January 15, 2024', reason: 'non-ISO format' }, { value: '01/15/2024', reason: 'US format slashes' }, { value: '2024/01/15', reason: 'forward slashes' }, { value: 'not-a-date', reason: 'non-date string' }, { value: '9999-12-31T23:59:59.999Z', reason: 'far future date' }, { value: '0000-01-01', reason: 'year zero' }, ]; for (const { value, reason } of invalidDates) { cases.push({ name: `${field.name}: should reject invalid date -- ${reason}`, field: field.name, category: 'format_violation', input: value, expectedError: { status: 400, code: 'VALIDATION_ERROR', }, }); } } if (field.type === 'uuid') { const invalidUuids = [ { value: 'not-a-uuid', reason: 'non-UUID string' }, { value: '12345678-1234-1234-1234-12345678901', reason: 'too short' }, { value: '12345678-1234-1234-1234-1234567890123', reason: 'too long' }, { value: '12345678123412341234123456789012', reason: 'missing hyphens' }, { value: 'ZZZZZZZZ-ZZZZ-ZZZZ-ZZZZ-ZZZZZZZZZZZZ', reason: 'non-hex characters' }, { value: '00000000-0000-0000-0000-000000000000', reason: 'nil UUID (may be invalid in context)' }, ]; for (const { value, reason } of invalidUuids) { cases.push({ name: `${field.name}: should reject invalid UUID -- ${reason}`, field: field.name, category: 'format_violation', input: value, expectedError: { status: 400, code: 'VALIDATION_ERROR', }, }); } } if (field.type === 'enum' && field.enumValues) { cases.push({ name: `${field.name}: should reject value not in enum`, field: field.name, category: 'format_violation', input: 'INVALID_ENUM_VALUE', expectedError: { status: 400, code: 'VALIDATION_ERROR', messagePattern: new RegExp(`${field.name}.*(?:invalid|must be one of|enum)`, 'i'), }, }); // Case sensitivity if (field.enumValues.length > 0) { const firstValue = field.enumValues[0]; cases.push({ name: `${field.name}: should reject wrong-case enum value`, field: field.name, category: 'format_violation', input: firstValue === firstValue.toLowerCase() ? firstValue.toUpperCase() : firstValue.toLowerCase(), expectedError: { status: 400, code: 'VALIDATION_ERROR' }, }); } } return cases; } ``` ## Injection Payload Generator ```typescript // injection-payloads.ts function generateInjectionPayloads(field: FieldDefinition): NegativeTestCase[] { const cases: NegativeTestCase[] = []; if (field.type === 'string' || field.type === 'email' || field.type === 'url') { const injectionStrings = [ // SQL injection { value: "'; DROP TABLE users; --", category: 'SQL injection' }, { value: "' OR '1'='1", category: 'SQL injection' }, { value: "' UNION SELECT * FROM passwords --", category: 'SQL injection' }, { value: "1; UPDATE users SET role='admin' WHERE '1'='1", category: 'SQL injection' }, // XSS { value: '', category: 'XSS' }, { value: '', category: 'XSS' }, { value: '">', category: 'XSS' }, { value: "javascript:alert('XSS')", category: 'XSS' }, { value: '', category: 'XSS' }, // Path traversal { value: '../../../etc/passwd', category: 'Path traversal' }, { value: '..\\..\\..\\windows\\system32\\config\\sam', category: 'Path traversal' }, { value: '/etc/shadow', category: 'Path traversal' }, // Command injection { value: '; ls -la', category: 'Command injection' }, { value: '| cat /etc/passwd', category: 'Command injection' }, { value: '$(whoami)', category: 'Command injection' }, { value: '`id`', category: 'Command injection' }, // LDAP injection { value: '*)(objectClass=*)', category: 'LDAP injection' }, // XML injection { value: ']>&xxe;', category: 'XXE' }, // Header injection { value: 'value\r\nInjected-Header: malicious', category: 'Header injection' }, // Unicode and encoding attacks { value: '\u0000null_byte', category: 'Null byte injection' }, { value: '%00null_byte', category: 'URL-encoded null byte' }, { value: '\uFEFFBOM_prefix', category: 'BOM injection' }, ]; for (const { value, category } of injectionStrings) { cases.push({ name: `${field.name}: should safely handle ${category} attempt`, field: field.name, category: 'injection', input: value, expectedError: { status: 400, code: 'VALIDATION_ERROR', // The response should NOT contain the injected content reflected back }, }); } } return cases; } ``` ## API Negative Testing ```typescript // create-user.negative.test.ts import { describe, it, expect, beforeAll, afterAll } from 'vitest'; const API_BASE = process.env.API_BASE_URL || 'http://localhost:3000/api'; // Define the schema for test generation const userFields: FieldDefinition[] = [ { name: 'email', type: 'email', required: true, maxLength: 255 }, { name: 'name', type: 'string', required: true, minLength: 1, maxLength: 100 }, { name: 'age', type: 'number', required: false, min: 0, max: 150 }, { name: 'role', type: 'enum', required: true, enumValues: ['admin', 'user', 'moderator'] }, { name: 'website', type: 'url', required: false, maxLength: 2048 }, { name: 'birthDate', type: 'date', required: false }, ]; const validUser = { email: 'test@example.com', name: 'Test User', age: 25, role: 'user', website: 'https://example.com', birthDate: '1998-06-15', }; describe('POST /api/users -- Negative Tests', () => { // Generate and run type violation tests for every field for (const field of userFields) { const typeViolations = generateTypeViolations(field); describe(`${field.name} -- type violations`, () => { for (const testCase of typeViolations) { it(testCase.name, async () => { const payload = { ...validUser, [field.name]: testCase.input }; const response = await fetch(`${API_BASE}/users`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(payload), }); expect(response.status).toBe(testCase.expectedError.status); const body = await response.json(); expect(body.code || body.error).toBeDefined(); // Verify error does not leak internal details const bodyText = JSON.stringify(body); expect(bodyText).not.toMatch(/stack|trace|node_modules|internal/i); }); } }); } // Missing required fields describe('Missing required fields', () => { const missingFieldCases = generateMissingFieldCases(userFields); for (const testCase of missingFieldCases) { it(testCase.name, async () => { const payload = { ...validUser }; if (testCase.input === undefined) { delete (payload as any)[testCase.field]; } else { (payload as any)[testCase.field] = testCase.input; } const response = await fetch(`${API_BASE}/users`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(payload), }); expect(response.status).toBe(400); }); } }); // Boundary violations describe('Boundary violations', () => { for (const field of userFields) { const boundaryViolations = generateBoundaryViolations(field); for (const testCase of boundaryViolations) { it(testCase.name, async () => { const payload = { ...validUser, [field.name]: testCase.input }; const response = await fetch(`${API_BASE}/users`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(payload), }); expect(response.status).toBe(400); }); } } }); // Format violations describe('Format violations', () => { for (const field of userFields) { const formatViolations = generateFormatViolations(field); for (const testCase of formatViolations) { it(testCase.name, async () => { const payload = { ...validUser, [field.name]: testCase.input }; const response = await fetch(`${API_BASE}/users`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(payload), }); expect(response.status).toBe(400); }); } } }); // Injection attempts describe('Injection attempts', () => { for (const field of userFields) { const injectionCases = generateInjectionPayloads(field); for (const testCase of injectionCases) { it(testCase.name, async () => { const payload = { ...validUser, [field.name]: testCase.input }; const response = await fetch(`${API_BASE}/users`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(payload), }); // Should either reject (400) or sanitize (200 with sanitized data) expect(response.status).toBeOneOf([400, 200, 422]); // If accepted, verify the injection is not reflected back unsanitized if (response.status === 200) { const body = await response.json(); const bodyStr = JSON.stringify(body); expect(bodyStr).not.toContain('