AWS S3 vs Hetzner's (S3-compatible) Object Storage¶
Hetzner only supports Object Storage located in eu-central — none in the US!
So, I'm curious about potential latency issues since both our users and servers
are primarily in the US. 🤔
I am performing benchmarking tests
using https://github.com/dvassallo/s3-benchmark to compare performance between
AWS S3 located in us-east-2 and Hetzner's Object Storage located in
eu-central (hel1).
Ideally, the benchmark is with us-east-1 instead of us-east-2. However,
us-east-1 is a
bit finicky. They're close
enough for purposes of this benchmark.
Test Setup¶
In this test, I have two variables:
- Storage location:
- AWS S3 (
us-east-2) - Hetzner Object Storage (
hel1)
- AWS S3 (
- Client location:
- Hetzner Server (
ash-dc1). Specs:- cpx11
- 2 vCPU
- 2 GB RAM
- 40 GB Disk local
- $4.99 /mo Price
- My own laptop to simulate a user in the US (Seattle)
- 2021 MacBook M1 Pro
- 32 GB RAM
- Hetzner Server (
Client location is an important variable because it represents where we will be accessing the storage from. With our Rails app, both the server and the user's browser accesses the storage. See sections below for specific use cases.
Server to Storage¶
Use cases:
- ActiveStorage uploads when not using Direct Uploads
- ActiveStorage processing variants
Benchmark Results¶
Hetzner Object Storage¶
Hetzner Server (ash-dc1) to Hetzner Object Storage (hel1)
Click to expand
--- BENCHMARK ----------------------------------------------------------------------------------------------------------------
Download performance with 1 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.0 MB/s | 144 104 105 106 130 247 359 435 | 157 104 105 126 150 288 400 435 |
| 9 | 0.0 MB/s | 156 104 105 106 158 329 384 420 | 169 104 105 108 158 336 420 425 |
| 10 | 0.1 MB/s | 164 104 105 106 118 370 375 400 | 174 104 105 112 157 370 388 400 |
| 11 | 0.1 MB/s | 156 104 105 107 128 323 362 373 | 166 105 105 114 150 326 373 396 |
| 12 | 0.1 MB/s | 154 104 105 107 116 344 376 397 | 161 104 106 113 148 348 376 445 |
| 13 | 0.1 MB/s | 137 104 105 106 116 259 336 346 | 150 104 106 124 148 259 336 346 |
| 14 | 0.1 MB/s | 159 104 105 109 179 314 361 380 | 176 104 106 148 180 319 408 420 |
| 15 | 0.1 MB/s | 148 104 106 110 154 259 364 376 | 166 105 109 148 160 272 364 420 |
| 16 | 0.1 MB/s | 164 104 105 107 205 292 375 423 | 186 104 106 149 243 332 416 424 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 2 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.1 MB/s | 117 104 104 105 112 135 208 349 | 138 104 105 142 151 160 252 350 |
| 9 | 0.1 MB/s | 123 104 105 105 113 147 287 299 | 143 104 106 147 151 160 287 299 |
| 10 | 0.1 MB/s | 141 104 105 107 112 310 353 366 | 167 105 107 147 152 316 353 408 |
| 11 | 0.1 MB/s | 135 104 105 106 110 265 344 386 | 147 104 105 109 148 265 388 428 |
| 12 | 0.1 MB/s | 125 104 105 106 114 125 332 341 | 132 104 106 107 122 158 366 383 |
| 13 | 0.1 MB/s | 128 104 105 107 114 127 362 366 | 139 105 105 113 147 163 362 366 |
| 14 | 0.2 MB/s | 130 104 105 110 119 154 348 363 | 148 104 107 123 153 177 388 408 |
| 15 | 0.1 MB/s | 129 104 105 106 116 200 327 329 | 138 104 105 111 147 200 328 329 |
| 16 | 0.2 MB/s | 135 104 105 110 117 268 317 412 | 156 104 109 145 151 301 328 456 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 4 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.2 MB/s | 164 104 105 105 232 306 358 359 | 181 106 107 148 267 319 360 362 |
| 9 | 0.1 MB/s | 183 105 105 108 262 364 435 442 | 198 106 107 153 272 374 437 442 |
| 10 | 0.2 MB/s | 175 104 105 109 236 337 421 421 | 188 106 107 129 260 355 423 463 |
| 11 | 0.2 MB/s | 171 104 105 106 199 356 392 423 | 183 106 107 110 201 382 396 464 |
| 12 | 0.2 MB/s | 158 104 105 106 190 273 398 399 | 179 106 108 149 203 306 401 440 |
| 13 | 0.2 MB/s | 151 104 105 107 124 274 401 403 | 175 106 108 148 172 300 403 448 |
| 14 | 0.2 MB/s | 151 104 105 108 147 284 354 382 | 171 105 109 148 187 285 355 426 |
| 15 | 0.2 MB/s | 154 104 105 114 118 287 398 419 | 174 106 107 116 156 320 460 533 |
| 16 | 0.3 MB/s | 149 104 105 110 128 289 351 355 | 161 106 107 121 150 326 356 363 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 8 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.3 MB/s | 165 104 105 106 145 352 527 537 | 185 108 109 149 156 360 542 568 |
| 9 | 0.4 MB/s | 144 104 105 106 118 258 375 382 | 162 108 109 128 154 265 379 429 |
| 10 | 0.5 MB/s | 141 104 105 106 116 270 399 410 | 152 108 109 111 148 273 414 448 |
| 11 | 0.5 MB/s | 134 104 105 105 112 225 353 380 | 151 107 108 119 151 225 383 395 |
| 12 | 0.5 MB/s | 148 104 105 109 119 321 373 391 | 165 108 109 128 152 324 376 437 |
| 13 | 0.5 MB/s | 151 104 105 109 132 293 355 383 | 163 107 108 115 154 313 386 387 |
| 14 | 0.5 MB/s | 162 104 105 108 133 342 406 409 | 178 107 108 136 151 383 412 415 |
| 15 | 0.4 MB/s | 151 104 105 107 122 320 375 513 | 164 106 108 119 151 323 384 515 |
| 16 | 0.5 MB/s | 156 104 105 111 126 315 391 425 | 176 106 114 147 156 329 396 427 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 16 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.7 MB/s | 142 104 105 105 123 295 388 397 | 146 108 109 109 127 299 392 401 |
| 9 | 0.8 MB/s | 141 104 105 107 116 286 359 369 | 146 108 109 111 123 290 364 373 |
| 10 | 0.8 MB/s | 153 104 105 109 139 305 346 367 | 157 108 109 114 143 309 350 371 |
| 11 | 0.9 MB/s | 148 104 105 107 121 292 406 413 | 152 108 109 111 131 295 409 416 |
| 12 | 0.9 MB/s | 147 104 105 106 133 292 347 378 | 151 107 109 111 136 295 350 381 |
| 13 | 1.0 MB/s | 155 104 105 110 131 327 375 377 | 160 107 108 114 136 329 378 380 |
| 14 | 0.9 MB/s | 158 104 105 108 129 373 406 438 | 161 107 108 110 132 376 409 441 |
| 15 | 1.2 MB/s | 144 104 106 113 139 247 328 335 | 147 106 108 120 139 248 329 337 |
| 16 | 1.1 MB/s | 156 105 106 109 122 295 368 404 | 158 106 108 112 124 297 370 406 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 32 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 1.2 MB/s | 193 104 105 117 252 347 611 720 | 203 109 110 120 263 395 616 725 |
| 9 | 1.3 MB/s | 170 104 105 110 241 316 402 437 | 175 109 110 118 244 320 406 442 |
| 10 | 1.4 MB/s | 172 104 105 109 233 320 363 366 | 176 109 110 114 238 324 365 371 |
| 11 | 1.5 MB/s | 181 104 105 111 256 356 423 442 | 186 108 109 120 261 359 426 446 |
| 12 | 1.9 MB/s | 167 104 105 109 221 301 390 402 | 171 108 109 113 224 304 394 406 |
| 13 | 1.8 MB/s | 184 104 106 116 291 352 370 423 | 187 107 109 120 294 355 373 427 |
| 14 | 1.8 MB/s | 187 105 106 117 289 345 411 451 | 190 107 108 120 292 348 413 454 |
| 15 | 2.1 MB/s | 175 104 106 111 268 338 385 391 | 178 107 108 114 270 341 387 394 |
| 16 | 2.1 MB/s | 178 104 107 110 249 343 390 407 | 181 107 109 113 251 347 391 408 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 64 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 3.1 MB/s | 143 104 105 105 127 282 309 394 | 147 109 109 110 133 286 313 398 |
| 9 | 3.0 MB/s | 150 104 105 108 132 300 360 376 | 155 109 110 111 135 304 364 381 |
| 10 | 3.6 MB/s | 158 104 105 107 123 332 382 393 | 162 107 108 112 127 335 387 397 |
| 11 | 3.4 MB/s | 154 104 105 106 128 323 360 367 | 158 107 108 109 131 325 363 370 |
| 12 | 4.0 MB/s | 150 104 105 109 128 291 405 407 | 153 107 108 112 135 294 407 409 |
| 13 | 4.6 MB/s | 140 104 105 106 128 245 320 344 | 144 107 108 109 132 247 323 347 |
| 14 | 5.1 MB/s | 144 104 106 109 130 274 327 351 | 147 107 108 112 134 276 330 353 |
| 15 | 4.4 MB/s | 146 104 106 112 126 280 366 367 | 150 107 108 116 132 282 369 369 |
| 16 | 4.9 MB/s | 139 104 105 109 130 230 349 358 | 142 106 107 112 133 231 351 359 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 128 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 4.7 MB/s | 173 104 106 116 205 330 442 470 | 177 109 110 121 209 341 446 475 |
| 9 | 6.0 MB/s | 147 105 105 111 125 283 341 346 | 152 109 109 119 129 287 346 356 |
| 10 | 6.3 MB/s | 147 105 105 106 121 303 388 420 | 151 108 109 110 125 306 391 423 |
| 11 | 6.6 MB/s | 157 105 105 109 126 340 392 413 | 161 108 109 113 131 343 395 417 |
| 12 | 7.4 MB/s | 146 104 105 109 124 250 383 438 | 150 108 109 112 131 253 386 441 |
| 13 | 8.9 MB/s | 153 105 106 109 121 313 404 414 | 157 108 110 112 123 316 407 417 |
| 14 | 6.9 MB/s | 163 104 106 109 127 340 420 455 | 168 107 108 116 134 342 423 457 |
| 15 | 8.6 MB/s | 162 105 107 113 155 311 349 389 | 166 108 110 118 157 314 352 392 |
| 16 | 7.9 MB/s | 163 105 106 114 134 332 403 425 | 166 107 109 117 137 336 406 427 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 256 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 13.0 MB/s | 132 105 105 107 121 160 342 382 | 137 110 110 113 126 165 346 387 |
| 9 | 15.1 MB/s | 124 105 106 107 115 137 289 296 | 129 109 110 112 122 142 293 300 |
| 10 | 14.0 MB/s | 131 105 106 110 115 145 400 406 | 137 108 111 115 124 148 405 412 |
| 11 | 13.4 MB/s | 134 105 106 107 119 160 367 410 | 139 109 109 110 127 163 371 413 |
| 12 | 19.0 MB/s | 134 105 106 108 118 179 351 364 | 137 108 110 112 126 182 354 367 |
| 13 | 17.8 MB/s | 148 105 106 110 116 273 361 391 | 153 108 109 113 127 276 364 393 |
| 14 | 14.9 MB/s | 168 105 107 113 205 360 424 512 | 184 108 112 131 218 374 428 515 |
| 15 | 16.5 MB/s | 158 105 106 110 145 328 362 383 | 166 108 109 126 156 331 373 386 |
| 16 | 22.1 MB/s | 152 105 106 112 149 261 351 369 | 158 108 110 117 152 288 354 372 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 512 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 22.5 MB/s | 155 106 106 107 140 317 433 608 | 167 112 112 115 158 323 444 658 |
| 9 | 21.5 MB/s | 158 106 108 116 142 302 425 451 | 176 112 117 135 183 340 431 457 |
| 10 | 20.7 MB/s | 174 105 121 134 152 335 398 438 | 202 119 140 154 229 350 413 453 |
| 11 | 24.8 MB/s | 181 106 125 146 179 340 412 447 | 201 124 143 178 199 357 430 466 |
| 12 | 23.7 MB/s | 199 108 144 188 199 288 361 414 | 220 126 163 214 217 304 385 431 |
| 13 | 23.0 MB/s | 206 106 144 162 231 343 449 519 | 235 137 178 184 248 364 467 540 |
| 14 | 22.7 MB/s | 232 108 196 215 268 342 414 503 | 252 145 214 233 287 359 431 520 |
| 15 | 23.6 MB/s | 229 109 158 199 287 350 396 486 | 266 138 206 252 305 372 414 503 |
| 16 | 23.9 MB/s | 243 108 172 217 270 378 449 648 | 280 128 217 252 287 412 468 665 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
AWS S3¶
Hetzner Server (ash-dc1) to AWS S3 (us-east-2)
Click to expand
--- BENCHMARK ----------------------------------------------------------------------------------------------------------------
Download performance with 1 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.2 MB/s | 44 30 32 34 35 97 114 127 | 44 30 32 34 35 97 114 127 |
| 9 | 0.2 MB/s | 45 29 32 34 40 86 100 110 | 45 29 32 34 40 86 100 110 |
| 10 | 0.2 MB/s | 45 29 32 34 40 93 104 108 | 45 29 32 34 40 93 104 108 |
| 11 | 0.2 MB/s | 46 30 32 34 37 97 102 116 | 46 30 32 34 37 97 102 116 |
| 12 | 0.2 MB/s | 49 31 33 35 45 100 106 113 | 49 31 33 35 45 100 106 113 |
| 13 | 0.2 MB/s | 52 31 34 36 66 103 112 113 | 52 31 34 36 66 103 112 113 |
| 14 | 0.2 MB/s | 52 31 34 36 69 103 111 117 | 52 31 34 36 69 103 111 118 |
| 15 | 0.2 MB/s | 54 31 34 36 72 104 112 136 | 54 31 34 36 72 104 112 136 |
| 16 | 0.3 MB/s | 57 31 34 37 79 109 116 121 | 57 31 34 37 79 109 116 121 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 2 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.3 MB/s | 44 29 33 35 37 70 127 134 | 44 29 33 35 37 70 127 134 |
| 9 | 0.4 MB/s | 43 30 33 35 36 68 97 98 | 43 30 33 35 36 68 97 98 |
| 10 | 0.4 MB/s | 44 29 32 33 35 90 101 105 | 44 29 32 33 35 90 101 105 |
| 11 | 0.4 MB/s | 48 30 33 35 40 98 102 102 | 48 30 33 35 40 98 102 102 |
| 12 | 0.5 MB/s | 48 31 34 35 41 100 102 104 | 48 31 34 35 41 100 102 104 |
| 13 | 0.5 MB/s | 51 30 33 35 68 99 104 110 | 51 30 33 35 68 99 104 110 |
| 14 | 0.5 MB/s | 54 32 34 36 68 105 120 125 | 54 32 34 36 68 105 120 125 |
| 15 | 0.5 MB/s | 51 29 33 36 70 97 100 102 | 51 29 33 36 70 97 100 102 |
| 16 | 0.5 MB/s | 53 30 32 35 80 104 107 114 | 53 30 32 35 80 104 107 114 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 4 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.7 MB/s | 45 30 33 35 37 92 101 151 | 45 30 33 35 37 92 101 151 |
| 9 | 0.8 MB/s | 45 31 33 35 37 71 115 116 | 45 31 33 35 37 72 115 116 |
| 10 | 0.8 MB/s | 45 30 33 34 38 90 100 121 | 45 30 33 34 38 90 100 121 |
| 11 | 0.8 MB/s | 49 30 33 35 37 100 116 118 | 49 31 33 35 37 100 116 118 |
| 12 | 0.9 MB/s | 50 30 33 36 42 102 107 114 | 50 30 33 36 42 103 107 114 |
| 13 | 0.9 MB/s | 50 31 33 34 38 99 109 111 | 50 31 33 34 38 99 109 111 |
| 14 | 1.0 MB/s | 48 30 32 34 65 92 97 99 | 48 30 32 34 65 92 97 99 |
| 15 | 1.0 MB/s | 51 31 33 34 68 95 101 111 | 51 31 33 34 68 95 101 111 |
| 16 | 0.9 MB/s | 54 29 32 35 89 98 102 112 | 54 29 32 35 89 98 102 112 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 8 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 1.2 MB/s | 47 31 34 35 37 99 113 123 | 47 31 34 35 37 99 113 123 |
| 9 | 1.4 MB/s | 46 31 33 35 36 86 117 129 | 46 31 33 35 36 86 117 129 |
| 10 | 1.6 MB/s | 47 30 32 34 44 96 103 108 | 47 30 32 34 44 96 103 108 |
| 11 | 1.6 MB/s | 48 31 33 35 41 102 112 128 | 49 31 33 35 41 102 112 128 |
| 12 | 1.8 MB/s | 48 31 33 35 42 101 105 108 | 49 31 33 35 42 101 105 108 |
| 13 | 1.9 MB/s | 50 30 32 35 57 98 101 105 | 50 30 32 35 57 98 101 105 |
| 14 | 2.0 MB/s | 50 31 32 35 67 98 107 108 | 50 31 33 35 67 98 107 108 |
| 15 | 2.1 MB/s | 52 31 34 36 68 100 107 110 | 52 31 34 36 68 100 107 110 |
| 16 | 2.0 MB/s | 56 30 33 35 87 107 114 115 | 56 30 33 35 87 107 114 115 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 16 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 2.8 MB/s | 43 32 33 34 36 83 105 117 | 43 32 33 34 36 83 105 117 |
| 9 | 2.8 MB/s | 46 32 34 35 37 93 107 108 | 46 32 34 35 37 93 107 108 |
| 10 | 3.2 MB/s | 46 30 34 35 40 94 101 110 | 46 30 34 35 40 94 101 110 |
| 11 | 3.5 MB/s | 46 32 34 36 37 89 101 102 | 46 32 34 36 37 89 101 102 |
| 12 | 3.7 MB/s | 46 31 33 34 45 91 96 100 | 47 31 33 34 45 91 96 100 |
| 13 | 3.7 MB/s | 50 31 34 36 38 98 106 107 | 50 31 34 36 38 98 107 108 |
| 14 | 3.7 MB/s | 53 31 34 36 66 102 121 126 | 53 31 34 36 67 102 121 126 |
| 15 | 4.1 MB/s | 51 31 33 35 70 97 106 109 | 51 31 33 35 70 97 107 109 |
| 16 | 4.5 MB/s | 52 31 33 35 70 99 101 103 | 52 31 33 35 70 99 101 103 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 32 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 5.2 MB/s | 43 29 32 34 36 69 112 118 | 46 30 34 35 37 83 132 135 |
| 9 | 5.7 MB/s | 44 30 32 34 39 68 101 109 | 48 31 33 35 52 87 111 129 |
| 10 | 5.5 MB/s | 49 29 33 35 37 103 147 150 | 53 30 33 35 38 124 152 170 |
| 11 | 6.4 MB/s | 46 28 32 34 41 97 108 108 | 51 29 33 35 55 115 128 129 |
| 12 | 6.0 MB/s | 48 30 32 34 54 96 103 103 | 54 30 33 35 68 112 122 123 |
| 13 | 6.6 MB/s | 49 29 33 34 49 100 103 112 | 55 30 34 35 68 118 120 132 |
| 14 | 7.3 MB/s | 49 30 32 34 69 97 101 104 | 55 31 33 35 70 114 120 123 |
| 15 | 7.3 MB/s | 53 29 32 35 68 107 111 114 | 60 30 33 36 69 127 129 134 |
| 16 | 7.5 MB/s | 54 30 33 35 76 105 111 112 | 62 31 34 36 96 125 132 132 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 64 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 9.7 MB/s | 42 29 32 35 37 82 101 109 | 50 30 33 36 38 82 140 144 |
| 9 | 10.8 MB/s | 42 31 32 34 36 72 92 97 | 49 31 33 35 39 108 128 137 |
| 10 | 11.2 MB/s | 43 29 32 33 36 92 96 108 | 52 30 33 35 62 111 132 145 |
| 11 | 11.7 MB/s | 47 31 32 34 36 98 108 113 | 55 32 34 35 38 136 148 153 |
| 12 | 12.2 MB/s | 47 29 33 34 38 95 101 108 | 57 30 34 35 67 131 143 149 |
| 13 | 13.1 MB/s | 47 29 31 33 51 91 100 101 | 57 30 32 35 69 126 140 141 |
| 14 | 12.9 MB/s | 51 30 32 34 65 105 110 110 | 63 31 34 35 76 138 149 152 |
| 15 | 13.0 MB/s | 52 30 33 36 68 105 109 114 | 66 31 34 37 86 144 149 154 |
| 16 | 14.2 MB/s | 52 30 32 35 70 101 102 105 | 66 31 34 37 89 140 142 145 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 128 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 16.1 MB/s | 51 30 35 38 42 119 152 152 | 59 31 37 40 44 155 189 192 |
| 9 | 17.7 MB/s | 50 30 37 40 43 96 112 112 | 60 31 39 42 53 138 156 170 |
| 10 | 20.6 MB/s | 46 28 34 37 40 94 99 105 | 57 29 36 39 53 132 147 152 |
| 11 | 22.2 MB/s | 49 31 35 39 42 96 105 107 | 59 32 36 41 55 130 146 148 |
| 12 | 21.3 MB/s | 52 30 38 40 44 99 104 110 | 65 31 39 43 64 140 154 170 |
| 13 | 21.2 MB/s | 53 29 35 39 47 101 112 112 | 67 30 36 45 78 147 154 155 |
| 14 | 23.9 MB/s | 53 31 36 38 57 98 106 125 | 68 32 37 40 79 149 162 166 |
| 15 | 23.4 MB/s | 55 31 35 41 57 103 109 112 | 72 33 39 50 88 145 150 172 |
| 16 | 25.5 MB/s | 56 28 35 40 91 104 108 109 | 73 31 39 44 99 146 151 163 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 256 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 31.2 MB/s | 51 31 35 38 42 118 145 151 | 61 32 36 39 53 178 201 205 |
| 9 | 35.3 MB/s | 47 31 34 38 44 92 103 107 | 60 32 36 41 58 120 168 186 |
| 10 | 37.7 MB/s | 49 28 34 38 41 100 110 112 | 63 30 37 40 53 155 170 171 |
| 11 | 36.8 MB/s | 49 28 35 40 44 92 100 102 | 66 30 39 43 63 147 158 162 |
| 12 | 40.6 MB/s | 52 30 35 39 45 104 109 111 | 69 32 38 44 64 159 172 172 |
| 13 | 40.7 MB/s | 53 31 35 40 49 101 105 108 | 72 34 38 47 77 157 167 172 |
| 14 | 35.3 MB/s | 58 30 36 39 76 110 119 134 | 82 32 38 58 105 166 178 181 |
| 15 | 44.0 MB/s | 55 31 34 38 72 105 116 117 | 77 33 37 53 108 155 177 187 |
| 16 | 46.3 MB/s | 55 29 35 39 83 99 110 110 | 78 30 40 53 125 160 170 170 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 512 KB objects (instance-type not found)
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 57.7 MB/s | 51 29 33 37 42 124 155 160 | 65 31 36 40 50 198 234 242 |
| 9 | 67.5 MB/s | 46 29 34 38 43 90 102 106 | 63 31 37 43 54 160 184 186 |
| 10 | 72.2 MB/s | 49 29 34 38 45 98 102 103 | 65 33 37 42 56 170 178 195 |
| 11 | 71.7 MB/s | 50 31 35 39 42 101 107 110 | 70 33 37 43 61 174 187 190 |
| 12 | 75.0 MB/s | 51 29 35 40 50 95 105 110 | 72 30 39 46 65 166 185 190 |
| 13 | 80.2 MB/s | 51 28 34 36 44 102 107 108 | 76 30 37 48 62 180 187 188 |
| 14 | 74.6 MB/s | 54 29 35 40 69 101 110 111 | 81 34 41 56 109 183 190 198 |
| 15 | 69.8 MB/s | 56 28 35 40 79 108 114 115 | 84 30 40 54 123 189 194 208 |
| 16 | 81.2 MB/s | 59 30 34 38 96 111 123 132 | 90 32 39 56 142 186 196 197 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
User (browser) to Storage¶
Use cases:
- Loading images on the website
- Profile pictures
- Organization logos/background images
- Uploading images via Direct Uploads
- This happens in the background, so latency is less of a concern
Benchmark Results¶
For these benchmarks, from my laptop, I noticed that the setup process for Hetzner took significantly longer than others. I've included the setup output which include the upload times.
Hetzner Object Storage¶
My laptop to Hetzner Object Storage (hel1)
Click to expand
--- SETUP --------------------------------------------------------------------------------------------------------------------
Uploading 1 KB objects
100% |████████████████████████████████████████| [8s:0s]
Uploading 2 KB objects
100% |████████████████████████████████████████| [10s:0s]
Uploading 4 KB objects
100% |████████████████████████████████████████| [7s:0s]
Uploading 8 KB objects
100% |████████████████████████████████████████| [13s:0s]
Uploading 16 KB objects
100% |████████████████████████████████████████| [29s:0s]
Uploading 32 KB objects
100% |████████████████████████████████████████| [58s:0s]
Uploading 64 KB objects
100% |████████████████████████████████████████| [1m34s:0s]
Uploading 128 KB objects
100% |████████████████████████████████████████| [2m19s:0s]
Uploading 256 KB objects
100% |████████████████████████████████████████| [3m51s:0s]
Uploading 512 KB objects
100% |████████████████████████████████████████| [29s:0s]
--- BENCHMARK ----------------------------------------------------------------------------------------------------------------
Download performance with 1 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.0 MB/s | 223 194 201 210 227 243 375 404 | 230 194 203 213 236 250 404 436 |
| 9 | 0.0 MB/s | 232 197 204 211 218 285 428 471 | 240 198 205 215 241 285 461 471 |
| 10 | 0.0 MB/s | 241 193 201 206 223 306 545 570 | 245 196 202 215 239 306 545 570 |
| 11 | 0.0 MB/s | 242 194 202 210 223 361 455 491 | 247 195 205 213 252 361 455 491 |
| 12 | 0.0 MB/s | 247 194 206 214 240 370 510 528 | 250 194 208 214 247 370 528 571 |
| 13 | 0.0 MB/s | 230 194 198 204 214 321 421 534 | 235 194 199 207 220 321 421 534 |
| 14 | 0.0 MB/s | 235 190 202 207 223 328 469 484 | 240 191 202 207 231 328 469 485 |
| 15 | 0.0 MB/s | 238 196 204 210 225 258 486 488 | 240 196 204 215 225 259 486 488 |
| 16 | 0.0 MB/s | 242 191 217 224 229 256 458 477 | 244 192 218 226 241 256 458 477 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 2 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.1 MB/s | 255 189 199 206 230 427 477 648 | 257 189 200 206 231 427 477 648 |
| 9 | 0.1 MB/s | 250 190 206 213 235 384 503 534 | 250 190 206 214 235 384 504 534 |
| 10 | 0.1 MB/s | 251 191 201 210 234 410 503 513 | 254 191 203 214 247 410 503 513 |
| 11 | 0.1 MB/s | 264 193 203 212 252 455 630 814 | 276 195 204 220 257 478 630 827 |
| 12 | 0.1 MB/s | 233 192 203 214 234 300 377 436 | 234 192 203 215 238 300 377 436 |
| 13 | 0.1 MB/s | 243 191 202 214 221 360 526 532 | 245 195 203 215 240 360 526 532 |
| 14 | 0.1 MB/s | 237 190 202 210 233 336 419 461 | 240 192 202 214 239 336 419 461 |
| 15 | 0.1 MB/s | 230 194 200 210 228 270 419 467 | 234 196 200 211 237 271 454 467 |
| 16 | 0.1 MB/s | 237 194 200 210 228 390 432 504 | 243 194 204 215 235 390 449 504 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 4 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.1 MB/s | 300 197 210 225 406 467 537 628 | 304 200 212 227 409 473 537 628 |
| 9 | 0.1 MB/s | 285 197 206 213 390 447 525 553 | 288 199 208 216 397 447 536 553 |
| 10 | 0.1 MB/s | 295 198 207 220 395 444 474 495 | 302 198 212 234 395 453 474 495 |
| 11 | 0.1 MB/s | 303 195 205 239 391 468 483 499 | 310 201 208 240 398 473 499 516 |
| 12 | 0.1 MB/s | 292 195 205 235 371 437 473 515 | 299 200 211 258 376 451 473 515 |
| 13 | 0.1 MB/s | 320 195 207 254 364 520 678 709 | 330 198 214 254 424 554 683 709 |
| 14 | 0.2 MB/s | 280 194 199 226 346 435 472 510 | 288 194 204 237 348 440 510 512 |
| 15 | 0.1 MB/s | 643 190 238 380 1285 1428 1546 1681 | 655 191 244 380 1288 1431 1546 1683 |
| 16 | 0.2 MB/s | 286 190 197 217 370 458 504 511 | 300 192 203 251 386 474 508 511 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 8 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.2 MB/s | 244 190 194 199 223 410 471 496 | 268 194 204 244 251 431 478 504 |
| 9 | 0.2 MB/s | 243 191 195 199 230 373 401 402 | 264 197 204 236 256 407 447 449 |
| 10 | 0.2 MB/s | 245 191 194 199 235 395 434 477 | 274 196 212 242 265 419 485 524 |
| 11 | 0.2 MB/s | 251 191 197 203 317 373 449 491 | 284 199 217 246 339 410 497 538 |
| 12 | 0.3 MB/s | 264 193 196 206 320 405 453 468 | 297 198 219 249 356 440 525 528 |
| 13 | 0.1 MB/s | 902 203 507 991 1161 1326 1452 1493 | 924 203 553 1024 1183 1333 1452 1541 |
| 14 | 0.3 MB/s | 249 188 197 205 224 410 458 485 | 274 197 208 244 273 420 485 495 |
| 15 | 0.1 MB/s | 714 190 201 237 542 2398 2551 2686 | 736 196 230 261 548 2402 2581 2691 |
| 16 | 0.3 MB/s | 396 199 351 384 415 575 598 669 | 415 201 362 405 431 583 632 669 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 16 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.5 MB/s | 210 190 198 203 215 232 262 274 | 219 196 206 210 223 241 277 286 |
| 9 | 0.3 MB/s | 428 195 203 363 557 808 978 1106 | 436 199 210 371 569 814 987 1113 |
| 10 | 0.4 MB/s | 312 189 202 328 387 439 505 517 | 320 196 210 334 398 444 513 521 |
| 11 | 0.6 MB/s | 267 192 201 209 330 396 573 578 | 274 202 209 218 334 409 573 583 |
| 12 | 0.4 MB/s | 411 193 211 368 545 760 795 908 | 419 202 214 374 554 766 803 916 |
| 13 | 0.4 MB/s | 519 195 323 387 566 973 1145 1292 | 525 198 333 396 576 978 1154 1299 |
| 14 | 0.3 MB/s | 609 196 389 463 993 1198 1227 1262 | 615 202 394 471 1006 1212 1227 1265 |
| 15 | 0.2 MB/s | 1141 194 493 1053 1593 1761 1803 2331 | 1146 203 501 1053 1596 1761 1808 2332 |
| 16 | 0.4 MB/s | 590 202 378 480 811 1012 1103 1115 | 596 202 383 485 823 1017 1106 1124 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 32 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.4 MB/s | 520 203 278 401 637 1213 1219 1257 | 531 213 289 410 650 1219 1237 1263 |
| 9 | 0.9 MB/s | 314 193 205 350 377 448 527 529 | 324 199 214 365 394 458 540 541 |
| 10 | 1.2 MB/s | 220 194 200 204 212 244 434 445 | 228 201 206 213 221 251 437 452 |
| 11 | 0.6 MB/s | 511 195 223 407 753 849 1007 1408 | 522 201 227 413 760 858 1016 1414 |
| 12 | 0.7 MB/s | 447 199 252 377 594 755 764 768 | 454 208 257 382 600 761 770 774 |
| 13 | 0.6 MB/s | 435 194 220 354 405 598 1524 1541 | 441 200 228 359 414 602 1534 1542 |
| 14 | 0.6 MB/s | 675 196 387 804 971 998 1010 1033 | 681 198 389 810 979 1002 1022 1037 |
| 15 | 0.7 MB/s | 579 242 385 462 865 972 1012 1014 | 585 253 389 467 867 979 1024 1024 |
| 16 | 0.8 MB/s | 544 195 427 558 635 790 850 890 | 551 202 433 582 642 796 852 895 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 64 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 1.3 MB/s | 332 191 198 345 416 518 559 589 | 344 202 208 352 424 520 567 603 |
| 9 | 1.3 MB/s | 425 194 221 360 434 861 930 998 | 434 202 229 378 442 869 937 1000 |
| 10 | 1.9 MB/s | 300 195 230 282 355 392 417 705 | 312 204 239 295 362 395 446 718 |
| 11 | 2.0 MB/s | 297 195 217 277 353 401 437 491 | 307 201 223 290 363 409 443 507 |
| 12 | 1.7 MB/s | 394 196 216 370 423 628 768 802 | 408 203 223 375 563 641 774 808 |
| 13 | 1.3 MB/s | 433 196 203 212 249 1632 1642 1651 | 443 202 210 224 278 1639 1649 1653 |
| 14 | 1.7 MB/s | 428 203 367 404 535 618 636 786 | 435 206 374 407 536 625 665 786 |
| 15 | 2.7 MB/s | 294 197 210 249 352 398 416 625 | 301 200 218 256 361 401 424 631 |
| 16 | 2.9 MB/s | 293 196 210 242 349 415 558 572 | 302 203 223 251 355 419 558 584 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 128 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 3.0 MB/s | 287 192 199 206 401 432 547 619 | 296 201 208 213 403 441 558 629 |
| 9 | 2.7 MB/s | 366 192 207 372 425 503 636 799 | 375 201 219 384 437 509 636 809 |
| 10 | 3.2 MB/s | 351 196 204 366 421 520 613 708 | 359 203 214 371 430 527 621 723 |
| 11 | 2.6 MB/s | 462 196 290 415 553 699 879 922 | 470 203 300 423 563 711 888 929 |
| 12 | 2.2 MB/s | 635 194 322 539 949 1022 1081 1113 | 644 200 333 545 952 1037 1088 1120 |
| 13 | 2.1 MB/s | 572 199 395 564 757 914 1147 1164 | 587 204 411 573 757 937 1154 1173 |
| 14 | 1.9 MB/s | 609 195 387 579 689 1064 1393 1396 | 620 199 397 588 720 1089 1399 1405 |
| 15 | 0.6 MB/s | 1687 192 228 476 583 9337 9576 9758 | 1694 202 239 491 595 9339 9576 9762 |
| 16 | 4.7 MB/s | 338 199 225 363 380 417 477 624 | 344 200 229 369 391 424 482 631 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 256 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 7.8 MB/s | 233 194 200 209 237 292 452 505 | 243 201 209 219 245 303 465 514 |
| 9 | 7.1 MB/s | 258 196 202 207 279 387 523 524 | 270 202 208 217 287 422 533 535 |
| 10 | 1.2 MB/s | 1960 199 370 532 1286 7375 7410 7414 | 1969 208 381 538 1293 7390 7415 7421 |
| 11 | 9.7 MB/s | 258 198 205 210 262 403 455 509 | 267 205 211 219 266 406 464 514 |
| 12 | 10.2 MB/s | 236 191 203 210 232 287 451 461 | 245 200 209 224 242 296 457 466 |
| 13 | 11.7 MB/s | 244 197 204 211 222 346 439 443 | 256 201 211 220 261 353 445 453 |
| 14 | 13.4 MB/s | 234 195 207 214 220 256 432 490 | 245 204 214 224 238 259 444 500 |
| 15 | 6.7 MB/s | 511 200 217 331 841 1177 1225 1237 | 521 210 227 335 866 1181 1231 1243 |
| 16 | 4.6 MB/s | 738 200 247 684 1253 1420 1581 1822 | 751 221 261 696 1256 1424 1585 1834 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 512 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 7.6 MB/s | 423 200 366 403 471 568 779 830 | 440 210 379 415 495 576 836 869 |
| 9 | 7.3 MB/s | 580 200 222 362 557 1656 1694 1729 | 593 207 230 369 564 1660 1701 1745 |
| 10 | 13.3 MB/s | 309 194 212 248 393 462 578 766 | 330 208 230 262 406 476 617 771 |
| 11 | 9.0 MB/s | 529 203 213 259 460 1389 1630 1654 | 552 214 222 307 531 1417 1641 1663 |
| 12 | 8.9 MB/s | 453 196 229 284 461 719 1502 1623 | 480 219 260 363 476 740 1514 1637 |
| 13 | 9.4 MB/s | 408 200 280 334 380 548 1373 1379 | 453 226 321 366 497 572 1394 1401 |
| 14 | 4.6 MB/s | 1396 364 1005 1272 1800 2083 3238 3445 | 1448 397 1046 1295 1831 2102 3262 3465 |
| 15 | 4.8 MB/s | 1177 442 692 1020 1694 1936 2335 2336 | 1215 459 707 1178 1740 1980 2364 2372 |
| 16 | 4.7 MB/s | 1515 357 616 1621 2106 2532 2676 2697 | 1540 388 649 1651 2131 2550 2694 2721 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
AWS S3¶
My laptop to AWS S3 (us-east-2)
Click to expand
--- SETUP --------------------------------------------------------------------------------------------------------------------
Uploading 1 KB objects
100% |████████████████████████████████████████| [2s:0s]
Uploading 2 KB objects
100% |████████████████████████████████████████| [2s:0s]
Uploading 4 KB objects
100% |████████████████████████████████████████| [2s:0s]
Uploading 8 KB objects
100% |████████████████████████████████████████| [2s:0s]
Uploading 16 KB objects
100% |████████████████████████████████████████| [2s:0s]
Uploading 32 KB objects
100% |████████████████████████████████████████| [2s:0s]
Uploading 64 KB objects
100% |████████████████████████████████████████| [3s:0s]
Uploading 128 KB objects
100% |████████████████████████████████████████| [3s:0s]
Uploading 256 KB objects
100% |████████████████████████████████████████| [3s:0s]
Uploading 512 KB objects
100% |████████████████████████████████████████| [10s:0s]
--- BENCHMARK ----------------------------------------------------------------------------------------------------------------
Download performance with 1 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.1 MB/s | 143 88 94 98 106 328 394 415 | 144 88 94 99 106 328 398 415 |
| 9 | 0.1 MB/s | 145 87 96 106 135 302 362 383 | 145 88 96 106 136 302 362 383 |
| 10 | 0.1 MB/s | 131 89 94 102 146 208 292 293 | 132 89 95 102 146 208 292 293 |
| 11 | 0.1 MB/s | 158 86 97 103 170 304 401 413 | 159 86 97 103 170 304 401 413 |
| 12 | 0.1 MB/s | 114 90 93 99 110 177 197 198 | 114 90 94 99 110 177 197 198 |
| 13 | 0.1 MB/s | 173 91 101 110 202 398 405 405 | 173 91 102 110 202 398 405 405 |
| 14 | 0.1 MB/s | 167 94 100 113 212 312 378 392 | 168 94 100 113 212 312 378 392 |
| 15 | 0.1 MB/s | 144 93 98 107 171 235 271 274 | 144 93 98 107 171 235 271 274 |
| 16 | 0.1 MB/s | 175 91 97 106 214 375 394 404 | 176 92 99 106 214 375 395 404 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 2 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.1 MB/s | 126 86 92 97 101 244 299 342 | 126 86 92 97 103 245 299 342 |
| 9 | 0.1 MB/s | 135 87 93 97 105 283 358 365 | 135 87 94 97 105 283 361 365 |
| 10 | 0.1 MB/s | 145 86 95 99 119 292 391 394 | 146 86 96 100 119 292 393 394 |
| 11 | 0.1 MB/s | 150 87 97 104 176 290 349 350 | 150 87 97 104 176 294 349 350 |
| 12 | 0.2 MB/s | 119 91 96 106 123 172 205 206 | 119 91 96 106 123 172 205 206 |
| 13 | 0.1 MB/s | 161 87 98 108 199 299 398 398 | 162 87 98 108 201 299 398 399 |
| 14 | 0.1 MB/s | 168 92 98 107 193 350 371 371 | 168 92 98 107 193 352 371 371 |
| 15 | 0.2 MB/s | 127 88 95 102 149 212 231 244 | 128 89 95 102 149 212 231 244 |
| 16 | 0.2 MB/s | 143 93 102 130 171 205 210 214 | 143 93 102 130 171 205 210 214 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 4 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.3 MB/s | 100 88 93 95 99 111 139 155 | 100 88 94 96 101 111 139 155 |
| 9 | 0.2 MB/s | 134 88 94 97 106 267 348 349 | 135 88 94 97 106 267 348 351 |
| 10 | 0.3 MB/s | 131 89 93 98 181 190 280 282 | 131 89 94 100 181 190 280 282 |
| 11 | 0.2 MB/s | 155 91 95 103 181 307 382 384 | 156 92 96 104 181 307 382 384 |
| 12 | 0.3 MB/s | 142 92 98 105 182 254 291 296 | 142 92 98 105 182 254 291 296 |
| 13 | 0.3 MB/s | 162 95 102 107 189 303 392 393 | 164 95 102 107 191 303 392 393 |
| 14 | 0.3 MB/s | 142 90 97 112 171 205 276 284 | 142 90 97 112 172 205 276 284 |
| 15 | 0.3 MB/s | 171 88 96 109 229 358 366 369 | 172 90 97 109 229 360 367 369 |
| 16 | 0.3 MB/s | 166 88 95 109 208 308 357 404 | 167 88 95 109 214 308 357 405 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 8 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 0.5 MB/s | 117 89 94 100 110 185 221 240 | 117 90 94 101 110 186 222 242 |
| 9 | 0.4 MB/s | 161 92 101 109 193 287 389 469 | 162 92 101 110 193 287 390 469 |
| 10 | 0.6 MB/s | 131 91 98 106 152 199 287 287 | 131 91 98 107 152 199 287 288 |
| 11 | 0.6 MB/s | 128 87 95 100 151 196 259 278 | 128 87 96 101 152 196 259 278 |
| 12 | 0.5 MB/s | 163 94 99 108 185 343 387 388 | 164 94 100 108 185 344 387 389 |
| 13 | 0.7 MB/s | 118 92 99 103 119 167 205 208 | 119 92 99 103 121 167 205 208 |
| 14 | 0.6 MB/s | 167 89 95 101 215 369 379 410 | 168 90 97 101 216 371 379 410 |
| 15 | 0.6 MB/s | 177 91 96 110 216 373 389 389 | 178 92 98 110 216 373 390 391 |
| 16 | 0.8 MB/s | 129 87 100 123 140 173 190 195 | 133 88 101 126 142 176 195 248 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 16 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 1.0 MB/s | 110 91 97 105 117 133 156 163 | 121 91 100 106 125 172 195 205 |
| 9 | 0.9 MB/s | 137 90 96 104 173 213 290 292 | 141 90 97 106 176 214 290 292 |
| 10 | 1.1 MB/s | 137 88 94 100 108 260 343 350 | 140 89 95 101 167 261 343 351 |
| 11 | 1.2 MB/s | 134 88 95 102 179 231 287 297 | 134 88 96 102 179 232 287 297 |
| 12 | 1.1 MB/s | 154 89 94 98 188 338 364 378 | 155 90 95 98 188 339 364 378 |
| 13 | 1.2 MB/s | 145 90 95 102 175 269 337 340 | 151 92 96 106 175 282 340 341 |
| 14 | 1.1 MB/s | 148 90 95 101 191 266 319 351 | 168 90 97 111 221 314 341 351 |
| 15 | 1.0 MB/s | 184 91 96 118 247 394 413 415 | 198 91 98 133 248 412 491 495 |
| 16 | 1.3 MB/s | 178 91 94 103 217 378 411 412 | 184 92 98 106 246 380 413 423 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 32 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 1.7 MB/s | 107 90 94 97 104 141 188 188 | 139 91 95 99 169 244 270 550 |
| 9 | 1.4 MB/s | 135 86 92 97 110 294 343 345 | 177 88 95 110 226 319 421 421 |
| 10 | 1.5 MB/s | 141 89 95 97 103 296 363 366 | 182 90 97 102 245 379 439 548 |
| 11 | 1.8 MB/s | 152 90 94 101 119 314 396 401 | 178 91 98 103 194 381 457 478 |
| 12 | 1.5 MB/s | 176 90 103 126 209 361 400 400 | 204 93 108 136 217 434 481 509 |
| 13 | 1.8 MB/s | 170 91 106 115 179 399 399 400 | 209 97 113 176 280 474 475 558 |
| 14 | 2.0 MB/s | 161 89 96 104 173 324 387 388 | 198 90 101 149 277 387 465 470 |
| 15 | 2.0 MB/s | 162 87 96 115 200 321 379 395 | 197 92 99 172 258 353 454 472 |
| 16 | 1.9 MB/s | 176 88 94 100 243 372 381 405 | 224 91 96 160 318 478 525 611 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 64 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 2.5 MB/s | 133 89 93 97 103 314 376 401 | 171 91 96 99 181 441 451 486 |
| 9 | 2.2 MB/s | 144 88 96 108 129 292 353 386 | 206 91 106 155 249 427 556 661 |
| 10 | 3.0 MB/s | 144 88 95 98 106 366 377 378 | 195 95 98 108 204 451 462 521 |
| 11 | 3.1 MB/s | 149 88 93 100 123 346 371 375 | 204 90 96 168 266 417 518 582 |
| 12 | 3.5 MB/s | 142 89 93 99 190 293 381 384 | 198 92 100 173 198 358 480 549 |
| 13 | 2.6 MB/s | 165 86 95 105 199 372 395 402 | 233 91 105 171 313 465 624 749 |
| 14 | 2.8 MB/s | 176 91 99 109 203 396 403 415 | 253 95 112 179 295 514 607 822 |
| 15 | 3.4 MB/s | 174 88 92 98 217 394 412 424 | 234 90 97 174 286 496 551 575 |
| 16 | 3.1 MB/s | 182 89 98 105 291 377 411 420 | 257 95 107 177 367 525 573 756 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 128 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 4.2 MB/s | 140 86 97 101 110 267 420 431 | 193 89 100 105 181 410 650 978 |
| 9 | 4.4 MB/s | 140 93 98 102 111 292 370 373 | 227 95 103 171 222 429 924 1365 |
| 10 | 4.8 MB/s | 149 95 99 104 123 306 384 386 | 221 97 110 176 281 473 552 626 |
| 11 | 3.3 MB/s | 160 95 100 105 136 331 424 431 | 305 97 104 114 354 868 996 1202 |
| 12 | 5.7 MB/s | 160 96 102 106 150 362 380 384 | 233 102 107 179 313 508 549 625 |
| 13 | 4.9 MB/s | 172 93 109 121 199 358 373 374 | 264 101 118 189 335 526 674 752 |
| 14 | 5.2 MB/s | 190 95 108 148 211 379 385 396 | 287 98 148 223 361 577 606 662 |
| 15 | 4.1 MB/s | 187 97 108 121 239 377 392 395 | 339 99 115 164 529 915 986 1068 |
| 16 | 4.2 MB/s | 182 93 99 104 369 387 402 404 | 355 96 106 186 536 936 1021 1022 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 256 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 6.6 MB/s | 148 91 101 105 112 360 429 441 | 253 95 104 109 201 657 1217 1448 |
| 9 | 8.8 MB/s | 143 90 99 105 114 274 376 399 | 229 98 104 112 264 465 728 1360 |
| 10 | 10.1 MB/s | 149 94 99 107 118 307 367 382 | 216 97 106 114 275 506 621 700 |
| 11 | 8.0 MB/s | 165 95 103 111 178 336 390 390 | 273 101 109 201 337 607 678 724 |
| 12 | 7.4 MB/s | 158 91 97 101 113 381 388 410 | 331 95 102 110 196 731 1477 1676 |
| 13 | 7.5 MB/s | 164 91 100 109 206 353 370 378 | 294 93 107 121 373 596 1653 1654 |
| 14 | 5.9 MB/s | 180 90 102 118 218 391 396 403 | 379 97 113 170 349 1299 1488 1562 |
| 15 | 8.5 MB/s | 173 87 99 106 114 377 386 445 | 312 92 105 181 576 684 1109 1468 |
| 16 | 12.5 MB/s | 188 94 105 123 270 370 373 373 | 287 98 112 169 430 622 719 721 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Download performance with 512 KB objects
+-------------------------------------------------------------------------------------------------+
| Time to First Byte (ms) | Time to Last Byte (ms) |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| Threads | Throughput | avg min p25 p50 p75 p90 p99 max | avg min p25 p50 p75 p90 p99 max |
+---------+----------------+------------------------------------------------+------------------------------------------------+
| 8 | 7.8 MB/s | 120 89 98 107 117 187 216 227 | 392 98 109 127 336 586 2414 2674 |
| 9 | 12.0 MB/s | 157 81 105 118 140 357 378 381 | 323 90 133 205 452 703 881 1204 |
| 10 | 7.1 MB/s | 156 92 102 110 132 373 382 390 | 345 108 121 167 314 813 2185 2201 |
| 11 | 8.5 MB/s | 150 88 95 101 110 367 374 380 | 447 98 104 110 209 2111 2275 2316 |
| 12 | 16.5 MB/s | 170 93 102 116 169 396 410 411 | 309 99 124 200 439 720 838 838 |
| 13 | 8.8 MB/s | 177 91 105 122 176 393 396 398 | 377 98 131 210 489 738 948 2310 |
| 14 | 10.7 MB/s | 173 93 100 109 177 377 387 389 | 400 102 111 390 511 805 824 1832 |
| 15 | 8.8 MB/s | 196 92 104 123 303 407 423 424 | 497 101 131 195 602 759 2211 2267 |
| 16 | 9.4 MB/s | 191 94 99 111 303 404 405 423 | 454 104 116 182 672 789 2083 2176 |
+---------+----------------+------------------------------------------------+------------------------------------------------+
Analyzed Results¶
https://docs.google.com/spreadsheets/d/1otN5-t1f7BhSFV5NrbBqDFT1WWLY_YYtc_YaDgA3Ndk/edit?usp=sharing
| First Byte (p90) | Last Byte (p90) | Thoughput | |
|---|---|---|---|
| AWS S3 to Hetzner Server | 97.10 ms | 118.95 ms | 15.75 MB/s |
| Hetzner Object Storage to Hetzner Server | 289.11 ms | 300.14 ms | 5.36 MB/s |
| AWS S3 to Macbook | 305.58 ms | 439.80 ms | 2.97 MB/s |
| Hetzner Object Storage to Macbook | 854.24 ms | 863.77 ms | 2.18 MB/s |
Switching our object storage from AWS (us-east-2) to Hetzner (hel1) will
result in:
- 297.75% increase in First Byte (p90) for our server
- 252.33% increase in Last Byte (p90) for our server
-
34.02% decrease in throughput for our server
-
279.55% increase in First Byte (p90) for web clients
- 196.40% increase in Last Byte (p90) for web clients
- 73.25% decrease in throughput for web clients
How to run benchmarks¶
- Follow install instructions: https://github.com/dvassallo/s3-benchmark
- Set up AWS/Hetzner S3
credentials: https://aws.amazon.com/blogs/security/a-new-and-standardized-way-to-manage-credentials-in-the-aws-sdks/
- Create a file called ~/.aws/credentials
- You can use AWS Profiles to store both the AWS and Hetzner credentials.
- Use
export AWS_PROFILE=your-profile-nameto set the profile before running the benchmark. - Alternatively, you can set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.
- Run benchmarks for Hetzner and AWS S3
export AWS_PROFILE=hetzner && ./s3-benchmark -region=hel1 -endpoint=https://hel1.your-objectstorage.com -upload-csv=dataexport AWS_PROFILE=aws && ./s3-benchmark -region=us-east-2 -upload-csv=data - Grab result CSV files from the buckets using the provider's dashboard.