Build failure promhippie/prometheus-hcloud-sd#29cde40e (master) by renovate-bot

Build failure promhippie/prometheus-vcd-sd#e71bdcdd (master) by tboerger

Hi folks,

I'm trying to build a DORA metrics dashboard based in the Worflows metrics (related issue) but I'm struggling to do it because I'm missing counters.

Before I go ahead, I would like to check if any of your tried to some something like that as I am trying to workaround the way the exporter is publishing the metrics, for example:
If a Job is ingested by the exporter, let's say: repo:MyRepo Workflow:Test Branch:main status:success run:100

The exporter creates a timeserie like:

github_workflow_status{..., repo:"MyRepo", workflow:"Test", branch:"main", run:"100", status:"success"}

and creates series every 30s (default ScrapeInterval) all the time like:
{..., repo:"MyRepo", workflow:"Test", branch:"main", run:"100", status:"success"} 7 @ timestamp
{..., repo:"MyRepo", workflow:"Test", branch:"main", run:"100", status:"success"} 7 @ timestamp (+ 30)
{..., repo:"MyRepo", workflow:"Test", branch:"main", run:"100", status:"success"} 7 @ timestamp (+ 30)
{..., repo:"MyRepo", workflow:"Test", branch:"main", run:"100", status:"success"} 7 @ timestamp (+30)

Until it reaches the Max Window time (default is 24h).

I hope I managed to explain the problem.

My question is:

Do you have any example how to count with these series format? As for example, I would like to know how many workflows runs succeed in a concrete period of time over a range (to create a dashboard with bars or line points) but with the problem that metrics keep reporting up to 24h I didn't mange.
I kind of sorted out reducing the windows period of time, but it doesn't seem to me a good way or checking instead github_workflow_updated_timestamp and searching by series with values between a range... but it is not ideal neither.

Probably I'm missing something on my PromQL so I would like to confirm if any of you saw and resolve something similar before keeping reinventing the wheel :)

Thank you

In reply to @dlouvier:matrix.org

Hi folks,

I'm trying to build a DORA metrics dashboard based in the Worflows metrics (related issue) but I'm struggling to do it because I'm missing counters.

Before I go ahead, I would like to check if any of your tried to some something like that as I am trying to workaround the way the exporter is publishing the metrics, for example:
If a Job is ingested by the exporter, let's say: repo:MyRepo Workflow:Test Branch:main status:success run:100

The exporter creates a timeserie like:

github_workflow_status{..., repo:"MyRepo", workflow:"Test", branch:"main", run:"100", status:"success"}

and creates series every 30s (default ScrapeInterval) all the time like:
{..., repo:"MyRepo", workflow:"Test", branch:"main", run:"100", status:"success"} 7 @ timestamp
{..., repo:"MyRepo", workflow:"Test", branch:"main", run:"100", status:"success"} 7 @ timestamp (+ 30)
{..., repo:"MyRepo", workflow:"Test", branch:"main", run:"100", status:"success"} 7 @ timestamp (+ 30)
{..., repo:"MyRepo", workflow:"Test", branch:"main", run:"100", status:"success"} 7 @ timestamp (+30)

Until it reaches the Max Window time (default is 24h).

I hope I managed to explain the problem.

My question is:

Do you have any example how to count with these series format? As for example, I would like to know how many workflows runs succeed in a concrete period of time over a range (to create a dashboard with bars or line points) but with the problem that metrics keep reporting up to 24h I didn't mange.
I kind of sorted out reducing the windows period of time, but it doesn't seem to me a good way or checking instead github_workflow_updated_timestamp and searching by series with values between a range... but it is not ideal neither.

Probably I'm missing something on my PromQL so I would like to confirm if any of you saw and resolve something similar before keeping reinventing the wheel :)

Thank you

Hi, I just wanted to know, I kind of solve this problem differently:

Something like this:
count by (repo,run) (github_workflow_status{status="in_progress",repo=~"$REPOSITORY"} unless (github_workflow_status{status="in_progress",repo=~"$REPOSITORY"} offset 3m)) or vector(0)

In reply to @tboerger:matrix.org
Oran Shuster: what kind of metric?

In reply to @tboerger:matrix.org
Before the integration of the webhooks I have kept it in workflow on purpose, since the webhooks are integrated it could be possible to implement job metrics. Maybe it makes sense to add another collector for these more detailed metrics.

you mean workflow metrics are not pulled but received through webhooks?
i know that these can be fetch using the github API as i wrote some python code to monitor them before we started using the exporter

In reply to @tboerger:matrix.org
Before the integration of the webhooks I have kept it in workflow on purpose, since the webhooks are integrated it could be possible to implement job metrics. Maybe it makes sense to add another collector for these more detailed metrics.

you mean workflow metrics are not pulled but received through webhooks?
i know that these can be fetch using the github API as i wrote some python code to monitor them before we started using the exporter

            runs = g.get_repo(full_name_or_id=repo_name).get_workflow_runs()
            jobs_dict = {"day": [], "job_title": [], "duration_seconds": [], "id": []}
            total_runs = 0

            for run in runs:
                if run.status not in ["success", "failure", "timed_out", "completed"]:
                    continue
                run_day = run.run_started_at
                jobs = run.jobs()
                for job in jobs: