[mdatagen] Add semconv reference for metrics in metadata schema by ChrsMark · Pull Request #13920 · open-telemetry/opentelemetry-collector

ChrsMark · 2025-09-30T13:41:42Z

Description

This PR adds support in mdatagen for defining SemConv reference for metrics.

Enhancing your metrics/signals in metadata.yaml like bellow:

metrics:
  system.cpu.time:
    enabled: true
    stability:
      level: beta
    description: Monotonic cumulative sum int metric enabled by default.
    unit: s
    sum:
      value_type: int
      monotonic: true
      aggregation_temporality: cumulative
    semantic_convention:
      semconv_ref: https://github.com/open-telemetry/semantic-conventions/blob/v1.37.0/docs/system/system-metrics.md#metric-systemcputime

Will provide the following generated docs:

system.cpu.time

Monotonic cumulative sum int metric enabled by default.

Unit	Metric Type	Value Type	Aggregation Temporality	Monotonic	Stability	Semantic Convention
s	Sum	Int	Cumulative	true	beta	system.cpu.time

Validation is added so as to ensure that provided SemConv urls match the top_level semconv version as well as the metric's name.

Link to tracking issue

Related to #13910 and #13297.

Testing

Tuned

Documentation

Tuned

codecov · 2025-09-30T13:55:36Z

Codecov Report

❌ Patch coverage is 94.05941% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.25%. Comparing base (3dbc211) to head (dafc640).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
cmd/mdatagen/internal/metric.go	77.77%	3 Missing and 3 partials ⚠️

❌ Your patch status has failed because the patch coverage (94.05%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #13920      +/-   ##
==========================================
+ Coverage   92.24%   92.25%   +0.01%     
==========================================
  Files         657      657              
  Lines       41153    41249      +96     
==========================================
+ Hits        37960    38053      +93     
- Misses       2186     2187       +1     
- Partials     1007     1009       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codeboten

i like this change a lot. would be great to get this in

ChrsMark · 2025-10-20T12:25:55Z

i like this change a lot. would be great to get this in

I'll get back into finalising this and opening for review soonish!

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>

ChrsMark · 2025-10-30T13:23:09Z

@codeboten @mx-psi I've opened this for review now. Feel free to take a look when you get the chance, thank's!

cmd/mdatagen/internal/metadata.go

mx-psi · 2025-10-30T13:40:32Z

cmd/mdatagen/internal/loader_test.go

 				GeneratedPackageName: "metadata",
 				Type:                 "sample",
-				SemConvVersion:       "1.9.0",
+				SemConvVersion:       "1.37.0",


I am not sure what this field is, but it seems like we should either drop this or make sure that sem_conv_version and the version linked in semconv_ref are aligned somehow

This is sth that already exists and links the whole component to a specific version of SemConv. It's not really used apart from a few places inside the templates that mdatagen populates i.e. logs.go.tmpl and metrics.go.tmpl.

The PR comes with a pattern checking that also verifies if the provided reference's version matches with this generic one: https://github.com/open-telemetry/opentelemetry-collector/pull/13920/files#diff-b33d4fa1a4595583d310c0a381ce3849aefb42419451584fd0e1c43bd16f5263R124-R128. (also covered by this test)

Okay, sorry, another thing I missed. I think that makes sense, though maybe we can remove it entirely if it's not widely used?

I am worried that we won't be able to support transition scenarios where we want to target two semconv versions

I am worried that we won't be able to support transition scenarios where we want to target two semconv versions

I think this deserves its own issue/discussion. So far the implementation implies that there is a single schema/semconv version for the whole metadata.yaml of a component. WDYT?

Also pinging @dmitryax and @braydonk for feedback here.

I think this deserves its own issue/discussion. So far the implementation implies that there is a single schema/semconv version for the whole metadata.yaml of a component. WDYT?

Oki, fair, we can leave it for later and merge this

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>

This patch adds the missing stability definition for the `elasticsearch.node.open_files` metric. Spotted at open-telemetry/opentelemetry-collector#13920. Signed-off-by: ChrsMark <chrismarkou92@gmail.com>

mx-psi

LGTM, since we were waiting on a few other people will wait a couple more days before merging

…43899) This patch adds the missing stability definition for the `elasticsearch.node.open_files` metric. Spotted at open-telemetry/opentelemetry-collector#13920. Signed-off-by: ChrsMark <chrismarkou92@gmail.com>

This PR contains the following updates: | Package | Type | Update | Change | Pending | |---|---|---|---|---| | [go.opentelemetry.io/collector/component](https://github.com/open-telemetry/opentelemetry-collector) | require | minor | `v1.45.0` → `v1.50.0` | `v1.51.0` | | [go.opentelemetry.io/collector/component/componenttest](https://github.com/open-telemetry/opentelemetry-collector) | require | minor | `v0.139.0` → `v0.144.0` | `v0.145.0` | | [go.opentelemetry.io/collector/confmap](https://github.com/open-telemetry/opentelemetry-collector) | require | minor | `v1.45.0` → `v1.50.0` | `v1.51.0` | | [go.opentelemetry.io/collector/consumer](https://github.com/open-telemetry/opentelemetry-collector) | require | minor | `v1.45.0` → `v1.50.0` | `v1.51.0` | | [go.opentelemetry.io/collector/consumer/consumertest](https://github.com/open-telemetry/opentelemetry-collector) | require | minor | `v0.139.0` → `v0.144.0` | `v0.145.0` | | [go.opentelemetry.io/collector/pdata](https://github.com/open-telemetry/opentelemetry-collector) | require | minor | `v1.45.0` → `v1.50.0` | `v1.51.0` | | [go.opentelemetry.io/collector/processor](https://github.com/open-telemetry/opentelemetry-collector) | require | minor | `v1.45.0` → `v1.50.0` | `v1.51.0` | | [go.opentelemetry.io/collector/processor/processortest](https://github.com/open-telemetry/opentelemetry-collector) | require | minor | `v0.139.0` → `v0.144.0` | `v0.145.0` | | [go.uber.org/zap](https://github.com/uber-go/zap) | require | patch | `v1.27.0` → `v1.27.1` | | --- ### Release Notes <details> <summary>open-telemetry/opentelemetry-collector (go.opentelemetry.io/collector/component)</summary> ### [`v1.50.0`](https://github.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v1500v01440) ##### 🛑 Breaking changes 🛑 - `pkg/exporterhelper`: Change verbosity level for otelcol\_exporter\_queue\_batch\_send\_size metric to detailed. ([#14278](open-telemetry/opentelemetry-collector#14278)) - `pkg/service`: Remove deprecated `telemetry.disableHighCardinalityMetrics` feature gate. ([#14373](open-telemetry/opentelemetry-collector#14373)) - `pkg/service`: Remove deprecated `service.noopTracerProvider` feature gate. ([#14374](open-telemetry/opentelemetry-collector#14374)) ##### 🚩 Deprecations 🚩 - `exporter/otlp_grpc`: Rename `otlp` exporter to `otlp_grpc` exporter and add deprecated alias `otlp`. ([#14403](open-telemetry/opentelemetry-collector#14403)) - `exporter/otlp_http`: Rename `otlphttp` exporter to `otlp_http` exporter and add deprecated alias `otlphttp`. ([#14396](open-telemetry/opentelemetry-collector#14396)) ##### 💡 Enhancements 💡 - `cmd/builder`: Avoid duplicate CLI error logging in generated collector binaries by relying on cobra's error handling. ([#14317](open-telemetry/opentelemetry-collector#14317)) - `cmd/mdatagen`: Add the ability to disable attributes at the metric level and re-aggregate data points based off of these new dimensions ([#10726](open-telemetry/opentelemetry-collector#10726)) - `cmd/mdatagen`: Add optional `display_name` and `description` fields to metadata.yaml for human-readable component names ([#14114](open-telemetry/opentelemetry-collector#14114)) The `display_name` field allows components to specify a human-readable name in metadata.yaml. When provided, this name is used as the title in generated README files. The `description` field allows components to include a brief description in generated README files. - `cmd/mdatagen`: Validate stability level for entities ([#14425](open-telemetry/opentelemetry-collector#14425)) - `pkg/xexporterhelper`: Reenable batching for profiles ([#14313](open-telemetry/opentelemetry-collector#14313)) - `receiver/nop`: add profiles signal support ([#14253](open-telemetry/opentelemetry-collector#14253)) ##### 🧰 Bug fixes 🧰 - `pkg/exporterhelper`: Fix reference count bug in partition batcher ([#14444](open-telemetry/opentelemetry-collector#14444))  ### [`v1.49.0`](https://github.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v1490v01430) ##### 💡 Enhancements 💡 - `all`: Update semconv import to 1.38.0 ([#14305](open-telemetry/opentelemetry-collector#14305)) - `exporter/nop`: Add profiles support to nop exporter ([#14331](open-telemetry/opentelemetry-collector#14331)) - `pkg/pdata`: Optimize the size and pointer bytes for pdata structs ([#14339](open-telemetry/opentelemetry-collector#14339)) - `pkg/pdata`: Avoid using interfaces/oneof like style for optional fields ([#14333](open-telemetry/opentelemetry-collector#14333))  ### [`v1.48.0`](https://github.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v1480v01420) ##### 💡 Enhancements 💡 - `exporter/debug`: Add logging of dropped attributes, events, and links counts in detailed verbosity ([#14202](open-telemetry/opentelemetry-collector#14202)) - `extension/memory_limiter`: The memorylimiter extension can be used as an HTTP/GRPC middleware. ([#14081](open-telemetry/opentelemetry-collector#14081)) - `pkg/config/configgrpc`: Statically validate gRPC endpoint ([#10451](open-telemetry/opentelemetry-collector#10451)) This validation was already done in the OTLP exporter. It will now be applied to any gRPC client. - `pkg/service`: Add support to disabling adding resource attributes as zap fields in internal logging ([#13869](open-telemetry/opentelemetry-collector#13869)) Note that this does not affect logs exported through OTLP.  ### [`v1.47.0`](https://github.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v1470v01410) ##### 🛑 Breaking changes 🛑 - `pkg/config/confighttp`: Use configoptional.Optional for confighttp.ClientConfig.Cookies field ([#14021](open-telemetry/opentelemetry-collector#14021)) ##### 💡 Enhancements 💡 - `pkg/config/confighttp`: Setting `compression_algorithms` to an empty list now disables automatic decompression, ignoring Content-Encoding ([#14131](open-telemetry/opentelemetry-collector#14131)) - `pkg/service`: Update semantic conventions from internal telemetry to v1.37.0 ([#14232](open-telemetry/opentelemetry-collector#14232)) - `pkg/xscraper`: Implement xscraper for Profiles. ([#13915](open-telemetry/opentelemetry-collector#13915)) ##### 🧰 Bug fixes 🧰 - `pkg/config/configoptional`: Ensure that configoptional.None values resulting from unmarshaling are equivalent to configoptional.Optional zero value. ([#14218](open-telemetry/opentelemetry-collector#14218))  ### [`v1.46.0`](https://github.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v1460v01400) ##### 💡 Enhancements 💡 - `cmd/mdatagen`: `metadata.yaml` now supports an optional `entities` section to organize resource attributes into logical entities with identity and description attributes ([#14051](open-telemetry/opentelemetry-collector#14051)) When entities are defined, mdatagen generates `AssociateWith{EntityType}()` methods on ResourceBuilder that associate resources with entity types using the entity refs API. The entities section is backward compatible - existing metadata.yaml files without entities continue to work as before. - `cmd/mdatagen`: Add semconv reference for metrics ([#13920](open-telemetry/opentelemetry-collector#13920)) - `connector/forward`: Add support for Profiles to Profiles ([#14092](open-telemetry/opentelemetry-collector#14092)) - `exporter/debug`: Disable sending queue by default ([#14138](open-telemetry/opentelemetry-collector#14138)) The recently added sending queue configuration in Debug exporter was enabled by default and had a problematic default size of 1. This change disables the sending queue by default. Users can enable and configure the sending queue if needed. - `pkg/config/configoptional`: Mark `configoptional.AddEnabledField` as beta ([#14021](open-telemetry/opentelemetry-collector#14021)) - `pkg/otelcol`: This feature has been improved and tested; secure-by-default redacts configopaque values ([#12369](open-telemetry/opentelemetry-collector#12369)) ##### 🧰 Bug fixes 🧰 - `all`: Ensure service service.instance.id is the same for all the signals when it is autogenerated. ([#14140](open-telemetry/opentelemetry-collector#14140))  </details> <details> <summary>uber-go/zap (go.uber.org/zap)</summary> ### [`v1.27.1`](https://github.com/uber-go/zap/releases/tag/v1.27.1) [Compare Source](uber-go/zap@v1.27.0...v1.27.1) Enhancements: - [#1501][]: prevent `Object` from panicking on nils - [#1511][]: Fix a race condition in `WithLazy`. Thanks to [@rabbbit](https://github.com/rabbbit), [@alshopov](https://github.com/alshopov), [@jquirke](https://github.com/jquirke), [@arukiidou](https://github.com/arukiidou) for their contributions to this release. [#1501]: uber-go/zap#1501 [#1511]: uber-go/zap#1511 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://gitea.t000-n.de/t.behrendt/tracebasedlogsampler/pulls/25 Reviewed-by: t.behrendt <t.behrendt@noreply.localhost> Co-authored-by: Renovate Bot <renovate@t00n.de> Co-committed-by: Renovate Bot <renovate@t00n.de>

ChrsMark mentioned this pull request Sep 30, 2025

Define guidelines for stability level of metrics #13910

Closed

ChrsMark changed the title ~~[mdatagen] Add semconv reference for signals in metadata schema~~ [mdatagen] Add semconv reference for metrics in metadata schema Sep 30, 2025

ChrsMark force-pushed the mdatagen_add_semconv_ref branch from ea7e141 to 51d5532 Compare September 30, 2025 15:09

ChrsMark mentioned this pull request Oct 1, 2025

[mdatagen] Consider linking metrics/attributes from metadata.yaml to SemConv definitions #13297

Open

codeboten reviewed Oct 10, 2025

View reviewed changes

ChrsMark force-pushed the mdatagen_add_semconv_ref branch 5 times, most recently from c46a42c to c156b07 Compare October 30, 2025 11:24

ChrsMark marked this pull request as ready for review October 30, 2025 11:27

ChrsMark requested review from a team and dmitryax as code owners October 30, 2025 11:27

github-actions bot added the cmd/mdatagen label Oct 30, 2025

ChrsMark mentioned this pull request Oct 30, 2025

[chore] add missing stability for es receiver metric open-telemetry/opentelemetry-collector-contrib#43899

Merged

[mdatagen] Add semconv ref for metrics

fe3a27c

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>

ChrsMark force-pushed the mdatagen_add_semconv_ref branch from c156b07 to fe3a27c Compare October 30, 2025 13:11

mx-psi reviewed Oct 30, 2025

View reviewed changes

tune naming

2442bf8

Signed-off-by: ChrsMark <chrismarkou92@gmail.com>

Merge branch 'main' into mdatagen_add_semconv_ref

dafc640

mx-psi approved these changes Nov 5, 2025

View reviewed changes

mx-psi added this pull request to the merge queue Nov 7, 2025

Merged via the queue into open-telemetry:main with commit 3b5e790 Nov 7, 2025
60 of 61 checks passed

github-actions bot added this to the next release milestone Nov 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[mdatagen] Add semconv reference for metrics in metadata schema#13920

[mdatagen] Add semconv reference for metrics in metadata schema#13920
mx-psi merged 3 commits intoopen-telemetry:mainfrom
ChrsMark:mdatagen_add_semconv_ref

ChrsMark commented Sep 30, 2025 •

edited

Loading

Uh oh!

codecov bot commented Sep 30, 2025 •

edited

Loading

Uh oh!

codeboten left a comment

Uh oh!

ChrsMark commented Oct 20, 2025

Uh oh!

ChrsMark commented Oct 30, 2025

Uh oh!

Uh oh!

Uh oh!

mx-psi Oct 30, 2025

Uh oh!

ChrsMark Oct 30, 2025

Uh oh!

mx-psi Oct 30, 2025

Uh oh!

ChrsMark Nov 3, 2025

Uh oh!

mx-psi Nov 3, 2025

Uh oh!

mx-psi left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ChrsMark commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

system.cpu.time

Link to tracking issue

Testing

Documentation

Uh oh!

codecov bot commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codeboten left a comment

Choose a reason for hiding this comment

Uh oh!

ChrsMark commented Oct 20, 2025

Uh oh!

ChrsMark commented Oct 30, 2025

Uh oh!

Uh oh!

Uh oh!

mx-psi Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

ChrsMark Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

mx-psi Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

ChrsMark Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

mx-psi Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

mx-psi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ChrsMark commented Sep 30, 2025 •

edited

Loading

codecov bot commented Sep 30, 2025 •

edited

Loading