Implement nested message flattening across all 4 generators (go-http, go-client, ts-client, openapiv3).

Purpose: Enable developers to flatten nested message fields to the parent level in JSON output. With flatten = true, child message fields are promoted to the parent object. With flatten_prefix, a prefix is prepended to avoid name collisions (e.g., billing_street, shipping_street). Single-level flatten only (no recursive flattening).

Output: Go generators produce custom MarshalJSON/UnmarshalJSON for messages with flatten-annotated fields. TypeScript interfaces include promoted child fields at the parent level. OpenAPI uses allOf to represent the flattened structure. Golden tests cover basic flatten, prefix flatten, and dual-flatten scenarios.

<execution_context> @/Users/sebastienmelki/.claude/get-shit-done/workflows/execute-plan.md @/Users/sebastienmelki/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/07-json-structural-transforms/07-RESEARCH.md @.planning/phases/07-json-structural-transforms/07-CONTEXT.md @.planning/phases/07-json-structural-transforms/07-01-SUMMARY.md

Existing encoding patterns to follow (established in Phase 4-6)

@internal/httpgen/encoding.go @internal/httpgen/nullable.go @internal/httpgen/timestamp_format.go

Annotation functions from Plan 01#

@internal/annotations/flatten.go

Generator entry points to modify#

@internal/httpgen/generator.go @internal/clientgen/generator.go @internal/tsclientgen/generator.go @internal/tsclientgen/types.go @internal/openapiv3/generator.go @internal/openapiv3/types.go

Task 1: Implement flatten in Go generators (go-http and go-client) internal/httpgen/flatten.go internal/httpgen/generator.go internal/clientgen/flatten.go internal/clientgen/generator.go Create internal/httpgen/flatten.go following the established encoding pattern:
  1. Define context structs:
type FlattenContext struct {
    Message      *protogen.Message
    FlattenInfos []*FlattenFieldInfo
}
 
type FlattenFieldInfo struct {
    Field  *protogen.Field
    Prefix string            // flatten_prefix value (may be empty)
}
  1. Implement detection and collection functions:
  • hasFlattenFields(message) bool -- checks if any field has flatten annotation
  • getFlattenFieldInfos(message) []*FlattenFieldInfo -- returns flatten fields with their prefixes
  • collectFlattenContexts(file) []*FlattenContext -- collects from all messages in file
  • validateFlattenAnnotations(file) error -- validates all flatten fields in file
    • For each message, for each field: call annotations.ValidateFlattenField
    • For each message with flatten fields: call annotations.ValidateFlattenCollisions
    • Check for MarshalJSON conflict: if message has BOTH flatten AND any of (int64 NUMBER encoding, nullable, empty_behavior, timestamp_format non-default, bytes_encoding non-default, oneof_config), emit error: "message X has both flatten and [other feature] on the same message -- this combination is not yet supported"
  1. Implement generateFlattenFile(file) error method on Generator:
  • File naming: file.GeneratedFilenamePrefix + "_flatten.pb.go"
  • Only generate if there are messages with flatten annotations
  • Run validation first; return error if validation fails
  1. Generated MarshalJSON pattern:
func (x *MessageName) MarshalJSON() ([]byte, error) {
    if x == nil {
        return []byte("null"), nil
    }
    data, err := protojson.Marshal(x)
    if err != nil {
        return nil, err
    }
    var raw map[string]json.RawMessage
    if err := json.Unmarshal(data, &raw); err != nil {
        return nil, err
    }
 
    // For each flattened field:
    // 1. Remove the nested field from raw
    // 2. Marshal the child message (use json.Marshal to respect child's MarshalJSON)
    // 3. Add each child field to raw with optional prefix
 
    if x.FlattenedFieldGoName != nil {
        delete(raw, "flattenedFieldJsonName")
        childData, err := json.Marshal(x.FlattenedFieldGoName)
        if err != nil {
            return nil, err
        }
        var childRaw map[string]json.RawMessage
        if err := json.Unmarshal(childData, &childRaw); err != nil {
            return nil, err
        }
        for k, v := range childRaw {
            raw["prefix"+k] = v  // prefix may be empty
        }
    }
 
    return json.Marshal(raw)
}

IMPORTANT: Use json.Marshal(x.FlattenedFieldGoName) to marshal the child, NOT protojson.Marshal(). This ensures that if the child message has its own MarshalJSON (for int64 encoding, timestamps, etc.), it gets invoked. This is the key to annotation composability as stated in CONTEXT.md.

  1. Generated UnmarshalJSON pattern:
func (x *MessageName) UnmarshalJSON(data []byte) error {
    var raw map[string]json.RawMessage
    if err := json.Unmarshal(data, &raw); err != nil {
        return err
    }
 
    // For each flattened field:
    // 1. Extract child fields from raw into a new map (with prefix stripped)
    // 2. Marshal the child map and unmarshal into the child message
    // 3. Delete the extracted fields from raw
 
    childRaw := make(map[string]json.RawMessage)
    // For each known child field of the flattened message:
    // Check if "prefix" + childJsonName exists in raw
    if v, ok := raw["prefixChildField1"]; ok {
        childRaw["childField1"] = v
        delete(raw, "prefixChildField1")
    }
    if v, ok := raw["prefixChildField2"]; ok {
        childRaw["childField2"] = v
        delete(raw, "prefixChildField2")
    }
    // ... for each child field
 
    if len(childRaw) > 0 {
        childData, err := json.Marshal(childRaw)
        if err != nil {
            return err
        }
        x.FlattenedFieldGoName = &ChildMessageType{}
        if err := json.Unmarshal(childData, x.FlattenedFieldGoName); err != nil {
            return err
        }
    }
 
    // Re-marshal remaining and unmarshal via protojson for non-flattened fields
    remaining, err := json.Marshal(raw)
    if err != nil {
        return err
    }
    return protojson.Unmarshal(remaining, x)
}

IMPORTANT for UnmarshalJSON: The child field JSON names must be known at generation time (they come from the child message's field descriptors). Generate a concrete list of field name checks, not a dynamic loop. Use json.Unmarshal(childData, x.FlattenedFieldGoName) to unmarshal the child, respecting its own UnmarshalJSON if present.

  1. Required imports for generated code: encoding/json, google.golang.org/protobuf/encoding/protojson

  2. In generator.go, add call to g.generateFlattenFile(file) after oneof_discriminator file call. Order: ..., bytes_encoding, oneof_discriminator, flatten.

  3. Copy internal/httpgen/flatten.go to internal/clientgen/flatten.go (change package name only). This follows the established identical-file pattern.

  4. In clientgen/generator.go, add the same call.

```bash go build ./internal/httpgen/... ./internal/clientgen/... go test -v ./internal/httpgen/... ./internal/clientgen/... make lint-fix ``` - internal/httpgen/flatten.go exists with FlattenContext and generation logic - internal/clientgen/flatten.go is identical (except package name) - generator.go in both packages calls generateFlattenFile - Validation detects flatten collisions and invalid field types - Validation detects MarshalJSON conflicts with other encoding features - Generated MarshalJSON promotes child fields to parent with optional prefix - Generated UnmarshalJSON extracts prefixed child fields and reconstructs child message - Child message annotation composability works (json.Marshal delegates to child MarshalJSON) - All existing golden tests still pass (backward compatible)
Task 2: Implement flatten in ts-client and openapiv3 with golden tests internal/tsclientgen/types.go internal/tsclientgen/generator.go internal/openapiv3/types.go internal/openapiv3/generator.go internal/httpgen/testdata/proto/flatten.proto internal/clientgen/testdata/proto/flatten.proto internal/tsclientgen/testdata/proto/flatten.proto internal/openapiv3/testdata/proto/flatten.proto **TypeScript client (internal/tsclientgen/types.go and generator.go):**

For a message with flattened fields, modify the interface generation:

  1. When generating the interface for a message with flatten fields, skip emitting the flattened field as a nested property
  2. Instead, inline the child message's fields at the parent level with the optional prefix
  3. Example:
// Without flatten:
export interface Order {
  id: string;
  billing: Address;
  shipping: Address;
}
 
// With flatten + prefix:
export interface Order {
  id: string;
  billing_street: string;
  billing_city: string;
  billing_zip: string;
  shipping_street: string;
  shipping_city: string;
  shipping_zip: string;
}

Implementation approach:

  1. In the interface generation function, check each field with annotations.IsFlattenField(field)
  2. For flatten fields, get the child message and iterate over its fields
  3. Apply annotations.GetFlattenPrefix(field) to each child field's JSON name
  4. Emit the child fields at the parent level with their types

OpenAPI (internal/openapiv3/types.go and generator.go):

For messages with flattened fields, modify the schema generation to use allOf:

Order:
  allOf:
    - type: object
      properties:
        id:
          type: string
    - type: object
      description: "billing (flattened with prefix billing_)"
      properties:
        billing_street:
          type: string
        billing_city:
          type: string
        billing_zip:
          type: string
    - type: object
      description: "shipping (flattened with prefix shipping_)"
      properties:
        shipping_street:
          type: string
        shipping_city:
          type: string
        shipping_zip:
          type: string

Alternative simpler approach: Generate a flat object schema with all properties at the top level (without using allOf). This is simpler and produces equivalent validation:

Order:
  type: object
  properties:
    id:
      type: string
    billing_street:
      type: string
    billing_city:
      type: string
    billing_zip:
      type: string
    shipping_street:
      type: string
    shipping_city:
      type: string
    shipping_zip:
      type: string

Choose the approach that best matches the existing OpenAPI generation patterns. If the codebase builds schemas property-by-property, the flat approach is easier. If it composes schemas, allOf is more semantic.

Golden test proto (flatten.proto):

Create the test proto in httpgen/testdata/proto/ and symlink to other generators:

syntax = "proto3";
package flatten.test;
 
import "sebuf/http/annotations.proto";
 
option go_package = "github.com/SebastienMelki/sebuf/internal/httpgen/testdata/flatten";
 
// Child message used for flattening
message Address {
  string street = 1;
  string city = 2;
  string zip = 3;
}
 
message ContactInfo {
  string email = 1;
  string phone = 2;
}
 
// Basic flatten without prefix
message SimpleFlatten {
  string id = 1;
  Address address = 2 [(sebuf.http.flatten) = true];
}
 
// Flatten with prefix (two flattened fields of same type)
message DualFlatten {
  string id = 1;
  Address billing = 2 [
    (sebuf.http.flatten) = true,
    (sebuf.http.flatten_prefix) = "billing_"
  ];
  Address shipping = 3 [
    (sebuf.http.flatten) = true,
    (sebuf.http.flatten_prefix) = "shipping_"
  ];
}
 
// Flatten with mixed fields (some flattened, some not)
message MixedFlatten {
  string id = 1;
  Address address = 2 [(sebuf.http.flatten) = true];
  ContactInfo contact = 3;  // Not flattened
  string notes = 4;
}
 
// No flatten annotation (backward compatible)
message PlainNested {
  string id = 1;
  Address address = 2;
}
 
service FlattenService {
  option (sebuf.http.service_config) = {
    base_path: "/api/v1"
  };
 
  rpc TestSimpleFlatten(SimpleFlatten) returns (SimpleFlatten) {
    option (sebuf.http.config) = {
      path: "/flatten/simple"
      method: HTTP_METHOD_POST
    };
  }
 
  rpc TestDualFlatten(DualFlatten) returns (DualFlatten) {
    option (sebuf.http.config) = {
      path: "/flatten/dual"
      method: HTTP_METHOD_POST
    };
  }
 
  rpc TestMixedFlatten(MixedFlatten) returns (MixedFlatten) {
    option (sebuf.http.config) = {
      path: "/flatten/mixed"
      method: HTTP_METHOD_POST
    };
  }
 
  rpc TestPlainNested(PlainNested) returns (PlainNested) {
    option (sebuf.http.config) = {
      path: "/flatten/plain"
      method: HTTP_METHOD_POST
    };
  }
}

Create symlinks from clientgen, tsclientgen, openapiv3 testdata/proto/ directories to the httpgen source proto.

Run UPDATE_GOLDEN=1 tests to generate golden files, then verify they pass without the flag:

UPDATE_GOLDEN=1 go test -run TestExhaustiveGoldenFiles ./internal/httpgen/...
UPDATE_GOLDEN=1 go test -run TestGoldenFiles ./internal/clientgen/...
UPDATE_GOLDEN=1 go test -run TestGoldenFiles ./internal/tsclientgen/...
UPDATE_GOLDEN=1 go test -run TestExhaustiveGoldenFiles ./internal/openapiv3/...
```bash go build ./internal/tsclientgen/... ./internal/openapiv3/... go test -v ./internal/httpgen/... ./internal/clientgen/... ./internal/tsclientgen/... ./internal/openapiv3/... make lint-fix ``` All golden tests pass without UPDATE_GOLDEN flag. Lint clean. - ts-client generates flattened interface types with promoted child fields - ts-client applies flatten_prefix to promoted field names - OpenAPI represents flattened structures correctly (allOf or flat properties) - PlainNested (no annotation) generates standard nested output (backward compatible) - DualFlatten with prefixes produces distinct field names - MixedFlatten shows both flattened and non-flattened fields - Golden test protos created and symlinked - All golden tests pass for all 4 generators - All existing tests pass (backward compatible)
1. Build passes: `go build ./...` 2. All tests pass: `go test ./...` 3. Lint clean: `make lint-fix` 4. Golden files updated and committed for all 4 generators 5. Go server and client produce identical MarshalJSON/UnmarshalJSON for flatten 6. TypeScript interfaces include promoted child fields with prefixes 7. OpenAPI schemas represent flattened structures correctly 8. Backward compatible: protos without flatten annotation produce identical output 9. Annotation composability: child message's own encoding annotations are respected through json.Marshal delegation

<success_criteria>

  • SimpleFlatten promotes Address fields (street, city, zip) to parent level in JSON
  • DualFlatten with prefixes produces billing_street, billing_city, shipping_street, shipping_city
  • MixedFlatten shows both flattened and non-flattened fields correctly
  • go-http and go-client produce identical code (byte-level after normalization)
  • TypeScript interfaces correctly inline flattened fields with prefixes
  • OpenAPI schemas accurately represent the flattened wire format
  • Generation-time error on flatten conflicts (collisions, invalid field types, MarshalJSON conflicts)
  • All golden tests pass, all existing tests pass </success_criteria>
After completion, create `.planning/phases/07-json-structural-transforms/07-03-SUMMARY.md`