13 Feb 2025 |
freephile | I do think cookies /can/ work in a browserless scenario - if the client supports cookies. But otherwise, you probably are getting to where you need to rely on HTTP headers with a request hash? | 15:49:58 |
Paladox | Doesn't seem to work | 16:00:24 |
Paladox | Still get: | 16:00:34 |
Paladox | Redacted or Malformed Event | 16:00:35 |
Paladox | /tmp/smw_chgprop_2pku0fk432lg_0.tmp is not readable. | 16:00:45 |
freephile | I just took a look at enable sticky sessions in haproxy and method #2 uses client IP rather than a cookie. Would that work in your environment? | 16:13:21 |
Paladox | I can see the cookie gets saved but each request won't pick it up since it'll act like a private browser. | 16:13:36 |
Paladox | well not saved but set | 16:13:44 |
Paladox | I guess option 2 could but then the concern is would it still load balance? Since it'll remember the IP. We run haproxy on the changeprop host and have it load localhost. | 16:15:07 |
freephile | I can't tell what the right solution is since it requires some analysis of the application architecture and request flows plus configuration options and DevOps policies for it all. | 16:22:52 |
freephile | Sorry I don't have time to look at the internals of your initial bug report to judge conclusively but it feels like it's an architecture issue, not a defect in the application. | 16:26:52 |
Paladox | I feel like the job should be re-worked because it looks like changeprop isn't made for sticky sessions. | 16:36:38 |
Paladox | I wonder if I could base sticky-sessions on x-request-id | 16:39:58 |
Paladox | hmm seems you can't do that in haproxy? only ip and cookie? | 16:49:38 |
Paladox | Neither of those 2 options seems to be adequate for this. Since you want the request to use the same host but it's not possible with how changeprop works. | 17:00:27 |
freephile | This is what Claude tells me:
global
log /dev/log local0
maxconn 4096
defaults
log global
mode http
option httplog
timeout connect 5000
timeout client 50000
timeout server 50000
frontend http_front
bind *:80
# Generate X-Request-ID if not present
http-request set-header X-Request-ID %[rand(0,1000000000)] if !{ req.hdr(X-Request-ID) -m found }
# Use consistent hashing based on X-Request-ID
default_backend http_back
backend http_back
balance consistent
hash-type consistent sdbm
# Use X-Request-ID for hash key
hash-header X-Request-ID
server server1 server1.example.com:80 check
server server2 server2.example.com:80 check
server server3 server3.example.com:80 check
# Preserve the X-Request-ID header
http-response set-header X-Request-ID %[req.hdr(X-Request-ID)]
| 18:20:17 |
Paladox | Oh, thanks! | 18:21:13 |
Paladox | i'll give that a go | 18:21:18 |
Paladox | i do round-robin, is that fine? | 18:21:32 |
freephile | I think consistent is different? But still gets you some balance?.... Ie, I'm guessing that "consistent" means "round robin, but stick" | 18:22:50 |
freephile | And, here's how Claude explains it:
This configuration:
- Generates a random X-Request-ID if one isn't present in the incoming request
- Uses consistent hashing based on the X-Request-ID header
- Ensures requests with the same X-Request-ID always go to the same backend server
- Preserves the X-Request-ID header in the response
Key features:
- Uses
hash-type consistent to minimize redistribution when servers are added/removed
sdbm hash algorithm provides good distribution
- Automatically generates IDs for requests that don't have them
- Maintains session stickiness based on the request ID
This is particularly useful when you need:
- Consistent routing for distributed tracing
- Debug capability across services
- Load balancing while maintaining request affinity
- Correlation of requests across multiple services
| 18:25:53 |
Paladox | https://github.com/miraheze/puppet/pull/4187/files | 18:27:22 |
Paladox | Redacted or Malformed Event | 18:30:31 |
Paladox | [ALERT] (2533095) : config : parsing [/etc/haproxy/conf.d/lb.cfg:28] : error detected in proxy 'jobrunner' while parsing 'http-response set-header' rule : sample fetch <req.hdr(X-Request-ID)]> may not be reliably used here because it needs 'HTTP request headers' which is not available here. | 18:30:33 |
Paladox | oh is it supposed to be http-request not http-response? | 18:31:18 |
Paladox | ChatGPT says i can do: | 18:34:22 |
Paladox | balance hdr(X-Request-ID)
hash-type consistent
| 18:34:23 |
Paladox | oh nvm the last part. | 18:34:46 |
Paladox | hmm says i can do the above | 18:35:44 |
freephile | I'm used to using frontend and backend over listen , and the set goes in the frontend if not already populated. That ChatGPT seems plausible | 18:35:49 |