[{"data":1,"prerenderedAt":1827},["ShallowReactive",2],{"post-\u002Fblog\u002Fcrc64nvme-new-integrity-default":3,"blog-index":1745},{"_path":4,"_dir":5,"_draft":6,"_partial":6,"_locale":7,"title":8,"description":9,"date":10,"author":11,"tags":12,"guide":16,"related":17,"body":20,"_type":1739,"_id":1740,"_source":1741,"_file":1742,"_stem":1743,"_extension":1744},"\u002Fblog\u002Fcrc64nvme-new-integrity-default","blog",false,"","CRC64NVME is now the S3 default, your SDK sends CRC32","S3 stores a CRC64NVME checksum on every new object, while the CLI and the SDKs default to different algorithms. Measured on a live endpoint.","2026-11-09","Clement Baranger",[13,14,15],"s3","integrity","api","s3-error-codes",[18,19],"multipart-5mb-floor-10000-parts","presigned-post-enforce-size-content-type",{"type":21,"children":22,"toc":1730},"root",[23,40,47,100,106,151,156,190,199,227,233,260,425,473,509,526,576,599,633,690,723,729,739,954,975,985,1124,1151,1223,1255,1273,1362,1382,1388,1423,1429,1513,1519,1724],{"type":24,"tag":25,"props":26,"children":27},"element","p",{},[28,31,38],{"type":29,"value":30},"text","A nightly backup starts answering ",{"type":24,"tag":32,"props":33,"children":35},"code",{"className":34},[],[36],{"type":29,"value":37},"400 BadDigest",{"type":29,"value":39},", and the message is about Content-MD5. No MD5 was ever sent: the SDK sends a base64 CRC now. Nothing in the repository changed between the last good run and the first bad one — a default did, and it decides which checksum travels with the object.",{"type":24,"tag":41,"props":42,"children":44},"h2",{"id":43},"the-short-version",[45],{"type":29,"value":46},"The short version",{"type":24,"tag":48,"props":49,"children":50},"ul",{},[51,57,62,83,95],{"type":24,"tag":52,"props":53,"children":54},"li",{},[55],{"type":29,"value":56},"Since 1 December 2024 S3 stores a CRC-based checksum on every new object, even when the client sends none; the algorithm it picks for itself is CRC64NVME.",{"type":24,"tag":52,"props":58,"children":59},{},[60],{"type":29,"value":61},"The algorithm the client sends is a separate decision, and the clients disagree: AWS CLI v2 defaults to CRC64NVME, while boto3, Go, Java and JavaScript default to CRC32. One bucket can hold both without anyone choosing.",{"type":24,"tag":52,"props":63,"children":64},{},[65,67,73,75,81],{"type":29,"value":66},"Reading the value back is opt-in: without ",{"type":24,"tag":32,"props":68,"children":70},{"className":69},[],[71],{"type":29,"value":72},"x-amz-checksum-mode: ENABLED",{"type":29,"value":74}," a ",{"type":24,"tag":32,"props":76,"children":78},{"className":77},[],[79],{"type":29,"value":80},"HEAD",{"type":29,"value":82}," returns no checksum at all, on an object that has one.",{"type":24,"tag":52,"props":84,"children":85},{},[86,88,93],{"type":29,"value":87},"A mismatch is ",{"type":24,"tag":32,"props":89,"children":91},{"className":90},[],[92],{"type":29,"value":37},{"type":29,"value":94},", whose documented text talks about Content-MD5 whatever you sent, and the SDK re-sends the whole request first — six attempts by default on one small object.",{"type":24,"tag":52,"props":96,"children":97},{},[98],{"type":29,"value":99},"The cost is CPU, not storage. A pure-Python CRC64NVME runs at 3.8 MiB\u002Fs on this machine against 5.86 GiB\u002Fs for the C library: 7.5 hours instead of 18 seconds for 100 GB.",{"type":24,"tag":41,"props":101,"children":103},{"id":102},"what-actually-happens",[104],{"type":29,"value":105},"What actually happens",{"type":24,"tag":25,"props":107,"children":108},{},[109,111,117,119,125,127,133,135,141,143,149],{"type":29,"value":110},"The checksum is computed by the client, before the first byte moves. It travels as a header named after the algorithm — ",{"type":24,"tag":32,"props":112,"children":114},{"className":113},[],[115],{"type":29,"value":116},"x-amz-checksum-crc64nvme",{"type":29,"value":118},", ",{"type":24,"tag":32,"props":120,"children":122},{"className":121},[],[123],{"type":29,"value":124},"x-amz-checksum-crc32",{"type":29,"value":126}," — with the 8-byte or 4-byte value base64-encoded, which is why a CRC64 looks like a twelve-character string such as ",{"type":24,"tag":32,"props":128,"children":130},{"className":129},[],[131],{"type":29,"value":132},"01qn\u002FOUzLVc=",{"type":29,"value":134},". As an ",{"type":24,"tag":32,"props":136,"children":138},{"className":137},[],[139],{"type":29,"value":140},"x-amz-",{"type":29,"value":142}," header on a signed request it joins ",{"type":24,"tag":32,"props":144,"children":146},{"className":145},[],[147],{"type":29,"value":148},"SignedHeaders",{"type":29,"value":150},", so a middlebox that rewrites it breaks the signature, not the checksum.",{"type":24,"tag":25,"props":152,"children":153},{},[154],{"type":29,"value":155},"On arrival S3 recomputes the value over the bytes it received and compares. Equal: the object is stored and the checksum goes into its metadata, re-readable months later. Not equal: nothing is stored.",{"type":24,"tag":25,"props":157,"children":158},{},[159,161,170,172,179,181,188],{"type":29,"value":160},"Since 1 December 2024 that happens whether or not your code participates: the ",{"type":24,"tag":162,"props":163,"children":167},"a",{"href":164,"rel":165},"https:\u002F\u002Faws.amazon.com\u002Fblogs\u002Faws\u002Fintroducing-default-data-integrity-protections-for-new-objects-in-amazon-s3\u002F",[166],"nofollow",[168],{"type":29,"value":169},"AWS announcement",{"type":29,"value":171}," says S3 computes a CRC and stores it for new objects even when the client sends nothing, and the ",{"type":24,"tag":162,"props":173,"children":176},{"href":174,"rel":175},"https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonS3\u002Flatest\u002Fuserguide\u002Fchecking-object-integrity-upload.html",[166],[177],{"type":29,"value":178},"user guide",{"type":29,"value":180}," adds that when neither you nor the SDK chooses an algorithm, S3 uses CRC64NVME. That page lists ten accepted algorithms, five ",{"type":24,"tag":162,"props":182,"children":185},{"href":183,"rel":184},"https:\u002F\u002Faws.amazon.com\u002Fabout-aws\u002Fwhats-new\u002F2026\u002F04\u002Fs3-five-additional-checksum-algorithms\u002F",[166],[186],{"type":29,"value":187},"added on 23 April 2026",{"type":29,"value":189},".",{"type":24,"tag":25,"props":191,"children":192},{},[193],{"type":24,"tag":194,"props":195,"children":198},"img",{"alt":196,"src":197},"Four steps of a signed upload — the client computes the CRC, Signature V4 covers the header, the server recomputes and compares before storing, and a read returns the value only when checksum mode is enabled — with a bottom band showing that the CLI sends CRC64NVME, the SDKs send CRC32, and a client that sends nothing gets CRC64NVME stored for it","\u002Fdiagrams\u002Fcrc64nvme-new-integrity-default.svg",[],{"type":24,"tag":25,"props":200,"children":201},{},[202,204,210,212,218,220,225],{"type":29,"value":203},"Whether the stored value can verify a file depends on the checksum type. For a multipart upload it is either ",{"type":24,"tag":32,"props":205,"children":207},{"className":206},[],[208],{"type":29,"value":209},"COMPOSITE",{"type":29,"value":211}," — the checksum of the concatenated part checksums, exactly like a multipart ETag — or ",{"type":24,"tag":32,"props":213,"children":215},{"className":214},[],[216],{"type":29,"value":217},"FULL_OBJECT",{"type":29,"value":219},", computed as if the object were one stream. Only the CRC family can linearise, so only CRC32, CRC32C and CRC64NVME support ",{"type":24,"tag":32,"props":221,"children":223},{"className":222},[],[224],{"type":29,"value":217},{"type":29,"value":226},", and a composite value cannot be compared with a CRC computed over the whole file.",{"type":24,"tag":41,"props":228,"children":230},{"id":229},"where-it-bites",[231],{"type":29,"value":232},"Where it bites",{"type":24,"tag":25,"props":234,"children":235},{},[236,238,244,246,252,254,259],{"type":29,"value":237},"I ran the cases against SeaweedFS 4.50 (",{"type":24,"tag":32,"props":239,"children":241},{"className":240},[],[242],{"type":29,"value":243},"weed server -s3",{"type":29,"value":245},", a container on this machine) with ",{"type":24,"tag":32,"props":247,"children":249},{"className":248},[],[250],{"type":29,"value":251},"boto3",{"type":29,"value":253}," 1.43.126. The object was 44,000 bytes, its CRC64NVME ",{"type":24,"tag":32,"props":255,"children":257},{"className":256},[],[258],{"type":29,"value":132},{"type":29,"value":189},{"type":24,"tag":261,"props":262,"children":263},"table",{},[264,283],{"type":24,"tag":265,"props":266,"children":267},"thead",{},[268],{"type":24,"tag":269,"props":270,"children":271},"tr",{},[272,278],{"type":24,"tag":273,"props":274,"children":275},"th",{},[276],{"type":29,"value":277},"Case",{"type":24,"tag":273,"props":279,"children":280},{},[281],{"type":29,"value":282},"What the endpoint did",{"type":24,"tag":284,"props":285,"children":286},"tbody",{},[287,322,347,365,384,402],{"type":24,"tag":269,"props":288,"children":289},{},[290,302],{"type":24,"tag":291,"props":292,"children":293},"td",{},[294,300],{"type":24,"tag":32,"props":295,"children":297},{"className":296},[],[298],{"type":29,"value":299},"put_object",{"type":29,"value":301}," with no checksum argument",{"type":24,"tag":291,"props":303,"children":304},{},[305,307,312,314,320],{"type":29,"value":306},"SDK sent ",{"type":24,"tag":32,"props":308,"children":310},{"className":309},[],[311],{"type":29,"value":124},{"type":29,"value":313}," on its own; ",{"type":24,"tag":32,"props":315,"children":317},{"className":316},[],[318],{"type":29,"value":319},"200",{"type":29,"value":321},", CRC32 echoed back",{"type":24,"tag":269,"props":323,"children":324},{},[325,330],{"type":24,"tag":291,"props":326,"children":327},{},[328],{"type":29,"value":329},"Correct CRC64NVME",{"type":24,"tag":291,"props":331,"children":332},{},[333,338,339,345],{"type":24,"tag":32,"props":334,"children":336},{"className":335},[],[337],{"type":29,"value":319},{"type":29,"value":118},{"type":24,"tag":32,"props":340,"children":342},{"className":341},[],[343],{"type":29,"value":344},"x-amz-checksum-crc64nvme: 01qn\u002FOUzLVc=",{"type":29,"value":346}," in the response",{"type":24,"tag":269,"props":348,"children":349},{},[350,355],{"type":24,"tag":291,"props":351,"children":352},{},[353],{"type":29,"value":354},"Wrong CRC64NVME",{"type":24,"tag":291,"props":356,"children":357},{},[358,363],{"type":24,"tag":32,"props":359,"children":361},{"className":360},[],[362],{"type":29,"value":37},{"type":29,"value":364}," — \"The Content-MD5 you specified did not match what we received.\"",{"type":24,"tag":269,"props":366,"children":367},{},[368,379],{"type":24,"tag":291,"props":369,"children":370},{},[371,377],{"type":24,"tag":32,"props":372,"children":374},{"className":373},[],[375],{"type":29,"value":376},"head_object",{"type":29,"value":378}," without checksum mode",{"type":24,"tag":291,"props":380,"children":381},{},[382],{"type":29,"value":383},"no checksum field in the response",{"type":24,"tag":269,"props":385,"children":386},{},[387,397],{"type":24,"tag":291,"props":388,"children":389},{},[390,392],{"type":29,"value":391},"Multipart, CRC64NVME, ",{"type":24,"tag":32,"props":393,"children":395},{"className":394},[],[396],{"type":29,"value":217},{"type":24,"tag":291,"props":398,"children":399},{},[400],{"type":29,"value":401},"completed, checksum equal to the CRC64 of the concatenated parts",{"type":24,"tag":269,"props":403,"children":404},{},[405,414],{"type":24,"tag":291,"props":406,"children":407},{},[408,409],{"type":29,"value":391},{"type":24,"tag":32,"props":410,"children":412},{"className":411},[],[413],{"type":29,"value":209},{"type":24,"tag":291,"props":415,"children":416},{},[417,423],{"type":24,"tag":32,"props":418,"children":420},{"className":419},[],[421],{"type":29,"value":422},"400 InvalidRequest",{"type":29,"value":424}," — \"Invalid Request\"",{"type":24,"tag":426,"props":427,"children":431},"pre",{"className":428,"code":429,"language":430,"meta":7,"style":7},"language-bash shiki shiki-themes github-light-high-contrast github-dark-high-contrast","python3 scripts\u002Fchecksum-probe.py probe http:\u002F\u002F127.0.0.1:8333 demo demosecret123\n","bash",[432],{"type":24,"tag":32,"props":433,"children":434},{"__ignoreMap":7},[435],{"type":24,"tag":436,"props":437,"children":440},"span",{"class":438,"line":439},"line",1,[441,447,453,458,463,468],{"type":24,"tag":436,"props":442,"children":444},{"style":443},"--shiki-default:#702C00;--shiki-dark:#FFB757",[445],{"type":29,"value":446},"python3",{"type":24,"tag":436,"props":448,"children":450},{"style":449},"--shiki-default:#032563;--shiki-dark:#ADDCFF",[451],{"type":29,"value":452}," scripts\u002Fchecksum-probe.py",{"type":24,"tag":436,"props":454,"children":455},{"style":449},[456],{"type":29,"value":457}," probe",{"type":24,"tag":436,"props":459,"children":460},{"style":449},[461],{"type":29,"value":462}," http:\u002F\u002F127.0.0.1:8333",{"type":24,"tag":436,"props":464,"children":465},{"style":449},[466],{"type":29,"value":467}," demo",{"type":24,"tag":436,"props":469,"children":470},{"style":449},[471],{"type":29,"value":472}," demosecret123\n",{"type":24,"tag":25,"props":474,"children":475},{},[476,482,484,490,492,499,501,507],{"type":24,"tag":477,"props":478,"children":479},"strong",{},[480],{"type":29,"value":481},"The error names an algorithm you did not use.",{"type":29,"value":483}," ",{"type":24,"tag":32,"props":485,"children":487},{"className":486},[],[488],{"type":29,"value":489},"BadDigest",{"type":29,"value":491}," is the documented code, and the ",{"type":24,"tag":162,"props":493,"children":496},{"href":494,"rel":495},"https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonS3\u002Flatest\u002FAPI\u002FAPI_Error.html",[166],[497],{"type":29,"value":498},"error catalogue",{"type":29,"value":500}," gives it one description: \"The Content-MD5 you specified did not match what we received.\" SeaweedFS returned that sentence verbatim for a CRC64NVME mismatch, so a reader who trusts the text hunts a ",{"type":24,"tag":32,"props":502,"children":504},{"className":503},[],[505],{"type":29,"value":506},"Content-MD5",{"type":29,"value":508}," header nobody has sent since 2024. Catch on the code instead.",{"type":24,"tag":25,"props":510,"children":511},{},[512,517,519,525],{"type":24,"tag":477,"props":513,"children":514},{},[515],{"type":29,"value":516},"The retry re-sends the object.",{"type":29,"value":518}," With the default retry configuration, botocore made six attempts at the 44,000-byte body before raising. A mismatch means the arriving bytes are not the sent bytes, so re-sending identical content is unlikely to help — but it happens anyway, and every attempt is a full payload. On a 100 MiB part that is 600 MiB on the wire, about 48 seconds per failed part at 100 Mbit\u002Fs: the same amplification as ",{"type":24,"tag":162,"props":520,"children":522},{"href":521},"\u002Fblog\u002Fmultipart-5mb-floor-10000-parts",[523],{"type":29,"value":524},"a multipart upload restarted instead of resumed",{"type":29,"value":189},{"type":24,"tag":25,"props":527,"children":528},{},[529,534,536,542,544,550,552,558,560,566,568,574],{"type":24,"tag":477,"props":530,"children":531},{},[532],{"type":29,"value":533},"The constant is easy to get wrong by one nibble.",{"type":29,"value":535}," CRC-64\u002FNVME is published as polynomial ",{"type":24,"tag":32,"props":537,"children":539},{"className":538},[],[540],{"type":29,"value":541},"0xAD93D23594C935A9",{"type":29,"value":543},", init all-ones, xorout all-ones, reflected in and out. Fed least-significant-bit first, the constant that reproduces the published vectors is ",{"type":24,"tag":32,"props":545,"children":547},{"className":546},[],[548],{"type":29,"value":549},"0x9A6C9329AC4BC9B5",{"type":29,"value":551},". Bit-reversing the published polynomial by hand gives ",{"type":24,"tag":32,"props":553,"children":555},{"className":554},[],[556],{"type":29,"value":557},"0x95AC9329AC4BC9B5",{"type":29,"value":559},", which yields ",{"type":24,"tag":32,"props":561,"children":563},{"className":562},[],[564],{"type":29,"value":565},"0x3558e8e979f60d7e",{"type":29,"value":567}," for ",{"type":24,"tag":32,"props":569,"children":571},{"className":570},[],[572],{"type":29,"value":573},"123456789",{"type":29,"value":575}," — a well-formed 8-byte checksum no endpoint will agree with, and nothing fails locally. Verify against the vectors first:",{"type":24,"tag":426,"props":577,"children":579},{"className":428,"code":578,"language":430,"meta":7,"style":7},"python3 scripts\u002Fchecksum-probe.py crc\n",[580],{"type":24,"tag":32,"props":581,"children":582},{"__ignoreMap":7},[583],{"type":24,"tag":436,"props":584,"children":585},{"class":438,"line":439},[586,590,594],{"type":24,"tag":436,"props":587,"children":588},{"style":443},[589],{"type":29,"value":446},{"type":24,"tag":436,"props":591,"children":592},{"style":449},[593],{"type":29,"value":452},{"type":24,"tag":436,"props":595,"children":596},{"style":449},[597],{"type":29,"value":598}," crc\n",{"type":24,"tag":426,"props":600,"children":603},{"className":601,"code":602,"language":29,"meta":7,"style":7},"language-text shiki shiki-themes github-light-high-contrast github-dark-high-contrast","input \"123456789\"    0xae8b14860a799888   rosUhgp5mIg=\ninput \"Hello World!\" 0x02e51cc85efce1a5   AuUcyF784aU=\ninput \"\"             0x0000000000000000   AAAAAAAAAAA=\n",[604],{"type":24,"tag":32,"props":605,"children":606},{"__ignoreMap":7},[607,615,624],{"type":24,"tag":436,"props":608,"children":609},{"class":438,"line":439},[610],{"type":24,"tag":436,"props":611,"children":612},{},[613],{"type":29,"value":614},"input \"123456789\"    0xae8b14860a799888   rosUhgp5mIg=\n",{"type":24,"tag":436,"props":616,"children":618},{"class":438,"line":617},2,[619],{"type":24,"tag":436,"props":620,"children":621},{},[622],{"type":29,"value":623},"input \"Hello World!\" 0x02e51cc85efce1a5   AuUcyF784aU=\n",{"type":24,"tag":436,"props":625,"children":627},{"class":438,"line":626},3,[628],{"type":24,"tag":436,"props":629,"children":630},{},[631],{"type":29,"value":632},"input \"\"             0x0000000000000000   AAAAAAAAAAA=\n",{"type":24,"tag":25,"props":634,"children":635},{},[636,641,643,648,650,655,657,664,666,673,675,681,683,688],{"type":24,"tag":477,"props":637,"children":638},{},[639],{"type":29,"value":640},"Providers implement a subset, and the failure is not always a checksum error.",{"type":29,"value":642}," Cloudflare R2 accepts CRC64NVME for ",{"type":24,"tag":32,"props":644,"children":646},{"className":645},[],[647],{"type":29,"value":217},{"type":29,"value":649}," only and CRC32 for ",{"type":24,"tag":32,"props":651,"children":653},{"className":652},[],[654],{"type":29,"value":209},{"type":29,"value":656}," only, ",{"type":24,"tag":162,"props":658,"children":661},{"href":659,"rel":660},"https:\u002F\u002Fdevelopers.cloudflare.com\u002Fr2\u002Fapi\u002Fs3\u002Fapi\u002F",[166],[662],{"type":29,"value":663},"per its compatibility table",{"type":29,"value":665},"; CRC64NVME arrived for single and multipart objects on ",{"type":24,"tag":162,"props":667,"children":670},{"href":668,"rel":669},"https:\u002F\u002Fdevelopers.cloudflare.com\u002Fr2\u002Fplatform\u002Frelease-notes\u002F",[166],[671],{"type":29,"value":672},"3 July 2025",{"type":29,"value":674},". Before that, the CLI's move broke uploads against R2 with ",{"type":24,"tag":32,"props":676,"children":678},{"className":677},[],[679],{"type":29,"value":680},"Header 'x-amz-checksum-algorithm' with value 'CRC32' not implemented",{"type":29,"value":682},". A header error surfaces before any data is validated, so a log grep for ",{"type":24,"tag":32,"props":684,"children":686},{"className":685},[],[687],{"type":29,"value":489},{"type":29,"value":689}," finds nothing.",{"type":24,"tag":25,"props":691,"children":692},{},[693,698,700,705,707,713,715,721],{"type":24,"tag":477,"props":694,"children":695},{},[696],{"type":29,"value":697},"Silence is the worst outcome.",{"type":29,"value":699}," An object stored with a CRC32 while the reader expects CRC64NVME is not an error anywhere: ",{"type":24,"tag":32,"props":701,"children":703},{"className":702},[],[704],{"type":29,"value":80},{"type":29,"value":706}," returns ",{"type":24,"tag":32,"props":708,"children":710},{"className":709},[],[711],{"type":29,"value":712},"ChecksumCRC32",{"type":29,"value":714},", the script looks for ",{"type":24,"tag":32,"props":716,"children":718},{"className":717},[],[719],{"type":29,"value":720},"ChecksumCRC64NVME",{"type":29,"value":722},", finds nothing, and passes.",{"type":24,"tag":41,"props":724,"children":726},{"id":725},"doing-it-right",[727],{"type":29,"value":728},"Doing it right",{"type":24,"tag":25,"props":730,"children":731},{},[732,737],{"type":24,"tag":477,"props":733,"children":734},{},[735],{"type":29,"value":736},"1. Choose the algorithm in the request, not in a comment.",{"type":29,"value":738}," Pass it explicitly so the version of the SDK stops deciding for you.",{"type":24,"tag":426,"props":740,"children":744},{"className":741,"code":742,"language":743,"meta":7,"style":7},"language-python shiki shiki-themes github-light-high-contrast github-dark-high-contrast","import boto3, base64\nfrom botocore.config import Config\n\ns3 = boto3.client(\"s3\", config=Config(request_checksum_calculation=\"when_supported\"))\ns3.put_object(Bucket=\"backups-prod\", Key=\"db\u002F2026-11-09.tar\", Body=body,\n              ChecksumAlgorithm=\"CRC64NVME\", ChecksumCRC64NVME=\"01qn\u002FOUzLVc=\")\n","python",[745],{"type":24,"tag":32,"props":746,"children":747},{"__ignoreMap":7},[748,763,785,794,855,914],{"type":24,"tag":436,"props":749,"children":750},{"class":438,"line":439},[751,757],{"type":24,"tag":436,"props":752,"children":754},{"style":753},"--shiki-default:#A0111F;--shiki-dark:#FF9492",[755],{"type":29,"value":756},"import",{"type":24,"tag":436,"props":758,"children":760},{"style":759},"--shiki-default:#0E1116;--shiki-dark:#F0F3F6",[761],{"type":29,"value":762}," boto3, base64\n",{"type":24,"tag":436,"props":764,"children":765},{"class":438,"line":617},[766,771,776,780],{"type":24,"tag":436,"props":767,"children":768},{"style":753},[769],{"type":29,"value":770},"from",{"type":24,"tag":436,"props":772,"children":773},{"style":759},[774],{"type":29,"value":775}," botocore.config ",{"type":24,"tag":436,"props":777,"children":778},{"style":753},[779],{"type":29,"value":756},{"type":24,"tag":436,"props":781,"children":782},{"style":759},[783],{"type":29,"value":784}," Config\n",{"type":24,"tag":436,"props":786,"children":787},{"class":438,"line":626},[788],{"type":24,"tag":436,"props":789,"children":791},{"emptyLinePlaceholder":790},true,[792],{"type":29,"value":793},"\n",{"type":24,"tag":436,"props":795,"children":797},{"class":438,"line":796},4,[798,803,808,813,818,822,827,831,836,841,845,850],{"type":24,"tag":436,"props":799,"children":800},{"style":759},[801],{"type":29,"value":802},"s3 ",{"type":24,"tag":436,"props":804,"children":805},{"style":753},[806],{"type":29,"value":807},"=",{"type":24,"tag":436,"props":809,"children":810},{"style":759},[811],{"type":29,"value":812}," boto3.client(",{"type":24,"tag":436,"props":814,"children":815},{"style":449},[816],{"type":29,"value":817},"\"s3\"",{"type":24,"tag":436,"props":819,"children":820},{"style":759},[821],{"type":29,"value":118},{"type":24,"tag":436,"props":823,"children":824},{"style":443},[825],{"type":29,"value":826},"config",{"type":24,"tag":436,"props":828,"children":829},{"style":753},[830],{"type":29,"value":807},{"type":24,"tag":436,"props":832,"children":833},{"style":759},[834],{"type":29,"value":835},"Config(",{"type":24,"tag":436,"props":837,"children":838},{"style":443},[839],{"type":29,"value":840},"request_checksum_calculation",{"type":24,"tag":436,"props":842,"children":843},{"style":753},[844],{"type":29,"value":807},{"type":24,"tag":436,"props":846,"children":847},{"style":449},[848],{"type":29,"value":849},"\"when_supported\"",{"type":24,"tag":436,"props":851,"children":852},{"style":759},[853],{"type":29,"value":854},"))\n",{"type":24,"tag":436,"props":856,"children":858},{"class":438,"line":857},5,[859,864,869,873,878,882,887,891,896,900,905,909],{"type":24,"tag":436,"props":860,"children":861},{"style":759},[862],{"type":29,"value":863},"s3.put_object(",{"type":24,"tag":436,"props":865,"children":866},{"style":443},[867],{"type":29,"value":868},"Bucket",{"type":24,"tag":436,"props":870,"children":871},{"style":753},[872],{"type":29,"value":807},{"type":24,"tag":436,"props":874,"children":875},{"style":449},[876],{"type":29,"value":877},"\"backups-prod\"",{"type":24,"tag":436,"props":879,"children":880},{"style":759},[881],{"type":29,"value":118},{"type":24,"tag":436,"props":883,"children":884},{"style":443},[885],{"type":29,"value":886},"Key",{"type":24,"tag":436,"props":888,"children":889},{"style":753},[890],{"type":29,"value":807},{"type":24,"tag":436,"props":892,"children":893},{"style":449},[894],{"type":29,"value":895},"\"db\u002F2026-11-09.tar\"",{"type":24,"tag":436,"props":897,"children":898},{"style":759},[899],{"type":29,"value":118},{"type":24,"tag":436,"props":901,"children":902},{"style":443},[903],{"type":29,"value":904},"Body",{"type":24,"tag":436,"props":906,"children":907},{"style":753},[908],{"type":29,"value":807},{"type":24,"tag":436,"props":910,"children":911},{"style":759},[912],{"type":29,"value":913},"body,\n",{"type":24,"tag":436,"props":915,"children":917},{"class":438,"line":916},6,[918,923,927,932,936,940,944,949],{"type":24,"tag":436,"props":919,"children":920},{"style":443},[921],{"type":29,"value":922},"              ChecksumAlgorithm",{"type":24,"tag":436,"props":924,"children":925},{"style":753},[926],{"type":29,"value":807},{"type":24,"tag":436,"props":928,"children":929},{"style":449},[930],{"type":29,"value":931},"\"CRC64NVME\"",{"type":24,"tag":436,"props":933,"children":934},{"style":759},[935],{"type":29,"value":118},{"type":24,"tag":436,"props":937,"children":938},{"style":443},[939],{"type":29,"value":720},{"type":24,"tag":436,"props":941,"children":942},{"style":753},[943],{"type":29,"value":807},{"type":24,"tag":436,"props":945,"children":946},{"style":449},[947],{"type":29,"value":948},"\"01qn\u002FOUzLVc=\"",{"type":24,"tag":436,"props":950,"children":951},{"style":759},[952],{"type":29,"value":953},")\n",{"type":24,"tag":25,"props":955,"children":956},{},[957,959,965,967,973],{"type":29,"value":958},"Telling the SDK to stop calculating at all is the other option — ",{"type":24,"tag":32,"props":960,"children":962},{"className":961},[],[963],{"type":29,"value":964},"when_required",{"type":29,"value":966},", or ",{"type":24,"tag":32,"props":968,"children":970},{"className":969},[],[971],{"type":29,"value":972},"AWS_REQUEST_CHECKSUM_CALCULATION=when_required",{"type":29,"value":974}," — right only when an endpoint rejects the header. Turning it off does not turn off what S3 stores.",{"type":24,"tag":25,"props":976,"children":977},{},[978,983],{"type":24,"tag":477,"props":979,"children":980},{},[981],{"type":29,"value":982},"2. Compute it once, on the way past.",{"type":29,"value":984}," The digest has to exist before the upload to be compared at all; a second read of the file afterwards measures the local copy twice and the network once.",{"type":24,"tag":426,"props":986,"children":988},{"className":741,"code":987,"language":743,"meta":7,"style":7},"from awscrt import checksums\nimport base64\n\nwith open(\"\u002Fbackups\u002Fdb-2026-11-09.tar\", \"rb\") as f:\n    digest = base64.b64encode(\n        checksums.crc64nvme(f.read()).to_bytes(8, \"big\")).decode()\n",[989],{"type":24,"tag":32,"props":990,"children":991},{"__ignoreMap":7},[992,1013,1025,1032,1080,1097],{"type":24,"tag":436,"props":993,"children":994},{"class":438,"line":439},[995,999,1004,1008],{"type":24,"tag":436,"props":996,"children":997},{"style":753},[998],{"type":29,"value":770},{"type":24,"tag":436,"props":1000,"children":1001},{"style":759},[1002],{"type":29,"value":1003}," awscrt ",{"type":24,"tag":436,"props":1005,"children":1006},{"style":753},[1007],{"type":29,"value":756},{"type":24,"tag":436,"props":1009,"children":1010},{"style":759},[1011],{"type":29,"value":1012}," checksums\n",{"type":24,"tag":436,"props":1014,"children":1015},{"class":438,"line":617},[1016,1020],{"type":24,"tag":436,"props":1017,"children":1018},{"style":753},[1019],{"type":29,"value":756},{"type":24,"tag":436,"props":1021,"children":1022},{"style":759},[1023],{"type":29,"value":1024}," base64\n",{"type":24,"tag":436,"props":1026,"children":1027},{"class":438,"line":626},[1028],{"type":24,"tag":436,"props":1029,"children":1030},{"emptyLinePlaceholder":790},[1031],{"type":29,"value":793},{"type":24,"tag":436,"props":1033,"children":1034},{"class":438,"line":796},[1035,1040,1046,1051,1056,1060,1065,1070,1075],{"type":24,"tag":436,"props":1036,"children":1037},{"style":753},[1038],{"type":29,"value":1039},"with",{"type":24,"tag":436,"props":1041,"children":1043},{"style":1042},"--shiki-default:#023B95;--shiki-dark:#91CBFF",[1044],{"type":29,"value":1045}," open",{"type":24,"tag":436,"props":1047,"children":1048},{"style":759},[1049],{"type":29,"value":1050},"(",{"type":24,"tag":436,"props":1052,"children":1053},{"style":449},[1054],{"type":29,"value":1055},"\"\u002Fbackups\u002Fdb-2026-11-09.tar\"",{"type":24,"tag":436,"props":1057,"children":1058},{"style":759},[1059],{"type":29,"value":118},{"type":24,"tag":436,"props":1061,"children":1062},{"style":449},[1063],{"type":29,"value":1064},"\"rb\"",{"type":24,"tag":436,"props":1066,"children":1067},{"style":759},[1068],{"type":29,"value":1069},") ",{"type":24,"tag":436,"props":1071,"children":1072},{"style":753},[1073],{"type":29,"value":1074},"as",{"type":24,"tag":436,"props":1076,"children":1077},{"style":759},[1078],{"type":29,"value":1079}," f:\n",{"type":24,"tag":436,"props":1081,"children":1082},{"class":438,"line":857},[1083,1088,1092],{"type":24,"tag":436,"props":1084,"children":1085},{"style":759},[1086],{"type":29,"value":1087},"    digest ",{"type":24,"tag":436,"props":1089,"children":1090},{"style":753},[1091],{"type":29,"value":807},{"type":24,"tag":436,"props":1093,"children":1094},{"style":759},[1095],{"type":29,"value":1096}," base64.b64encode(\n",{"type":24,"tag":436,"props":1098,"children":1099},{"class":438,"line":916},[1100,1105,1110,1114,1119],{"type":24,"tag":436,"props":1101,"children":1102},{"style":759},[1103],{"type":29,"value":1104},"        checksums.crc64nvme(f.read()).to_bytes(",{"type":24,"tag":436,"props":1106,"children":1107},{"style":1042},[1108],{"type":29,"value":1109},"8",{"type":24,"tag":436,"props":1111,"children":1112},{"style":759},[1113],{"type":29,"value":118},{"type":24,"tag":436,"props":1115,"children":1116},{"style":449},[1117],{"type":29,"value":1118},"\"big\"",{"type":24,"tag":436,"props":1120,"children":1121},{"style":759},[1122],{"type":29,"value":1123},")).decode()\n",{"type":24,"tag":25,"props":1125,"children":1126},{},[1127,1133,1135,1141,1143,1149],{"type":24,"tag":32,"props":1128,"children":1130},{"className":1129},[],[1131],{"type":29,"value":1132},"awscrt.checksums",{"type":29,"value":1134}," ships with the AWS CRT and Go's ",{"type":24,"tag":32,"props":1136,"children":1138},{"className":1137},[],[1139],{"type":29,"value":1140},"crc64",{"type":29,"value":1142}," has ",{"type":24,"tag":32,"props":1144,"children":1146},{"className":1145},[],[1147],{"type":29,"value":1148},"crc64nvme",{"type":29,"value":1150},"; a Python loop is 1,500× slower, and the fast path is already in the SDK you installed.",{"type":24,"tag":25,"props":1152,"children":1153},{},[1154,1166,1168,1174,1176,1182,1184,1190,1192,1197,1199,1205,1207,1213,1215,1221],{"type":24,"tag":477,"props":1155,"children":1156},{},[1157,1159,1164],{"type":29,"value":1158},"3. Ask for ",{"type":24,"tag":32,"props":1160,"children":1162},{"className":1161},[],[1163],{"type":29,"value":217},{"type":29,"value":1165}," when you intend to compare whole files.",{"type":29,"value":1167}," On ",{"type":24,"tag":32,"props":1169,"children":1171},{"className":1170},[],[1172],{"type":29,"value":1173},"CreateMultipartUpload",{"type":29,"value":1175},", set ",{"type":24,"tag":32,"props":1177,"children":1179},{"className":1178},[],[1180],{"type":29,"value":1181},"ChecksumAlgorithm=\"CRC64NVME\"",{"type":29,"value":1183}," with ",{"type":24,"tag":32,"props":1185,"children":1187},{"className":1186},[],[1188],{"type":29,"value":1189},"ChecksumType=\"FULL_OBJECT\"",{"type":29,"value":1191},". A ",{"type":24,"tag":32,"props":1193,"children":1195},{"className":1194},[],[1196],{"type":29,"value":209},{"type":29,"value":1198}," value answers a different question — whether the parts arrived intact — and cannot be compared to a CRC of the local file. Two 5 MiB parts of ",{"type":24,"tag":32,"props":1200,"children":1202},{"className":1201},[],[1203],{"type":29,"value":1204},"\\x01",{"type":29,"value":1206}," and ",{"type":24,"tag":32,"props":1208,"children":1210},{"className":1209},[],[1211],{"type":29,"value":1212},"\\x02",{"type":29,"value":1214}," returned ",{"type":24,"tag":32,"props":1216,"children":1218},{"className":1217},[],[1219],{"type":29,"value":1220},"XwQeSvWLj+8=",{"type":29,"value":1222},", the CRC64 of the concatenation, not of the part checksums.",{"type":24,"tag":25,"props":1224,"children":1225},{},[1226,1231,1232,1238,1240,1246,1248,1254],{"type":24,"tag":477,"props":1227,"children":1228},{},[1229],{"type":29,"value":1230},"4. Read the value back deliberately.",{"type":29,"value":483},{"type":24,"tag":32,"props":1233,"children":1235},{"className":1234},[],[1236],{"type":29,"value":1237},"head_object(..., ChecksumMode=\"ENABLED\")",{"type":29,"value":1239}," or ",{"type":24,"tag":32,"props":1241,"children":1243},{"className":1242},[],[1244],{"type":29,"value":1245},"get_object",{"type":29,"value":1247}," with the same argument; without it the field is absent. On download, compare the returned base64 with your own value, or let the SDK do it via ",{"type":24,"tag":32,"props":1249,"children":1251},{"className":1250},[],[1252],{"type":29,"value":1253},"response_checksum_validation",{"type":29,"value":189},{"type":24,"tag":25,"props":1256,"children":1257},{},[1258,1263,1265,1271],{"type":24,"tag":477,"props":1259,"children":1260},{},[1261],{"type":29,"value":1262},"5. Prove it on the endpoint you deploy against.",{"type":29,"value":1264}," The probe creates a bucket called ",{"type":24,"tag":32,"props":1266,"children":1268},{"className":1267},[],[1269],{"type":29,"value":1270},"crc-probe",{"type":29,"value":1272}," and runs the table above in one command:",{"type":24,"tag":426,"props":1274,"children":1276},{"className":428,"code":1275,"language":430,"meta":7,"style":7},"python3 scripts\u002Fchecksum-probe.py probe https:\u002F\u002F\u003Caccount>.r2.cloudflarestorage.com \u003Ckey> \u003Csecret>\n",[1277],{"type":24,"tag":32,"props":1278,"children":1279},{"__ignoreMap":7},[1280],{"type":24,"tag":436,"props":1281,"children":1282},{"class":438,"line":439},[1283,1287,1291,1295,1300,1305,1310,1315,1320,1325,1330,1335,1340,1344,1348,1353,1357],{"type":24,"tag":436,"props":1284,"children":1285},{"style":443},[1286],{"type":29,"value":446},{"type":24,"tag":436,"props":1288,"children":1289},{"style":449},[1290],{"type":29,"value":452},{"type":24,"tag":436,"props":1292,"children":1293},{"style":449},[1294],{"type":29,"value":457},{"type":24,"tag":436,"props":1296,"children":1297},{"style":449},[1298],{"type":29,"value":1299}," https:\u002F\u002F",{"type":24,"tag":436,"props":1301,"children":1302},{"style":753},[1303],{"type":29,"value":1304},"\u003C",{"type":24,"tag":436,"props":1306,"children":1307},{"style":449},[1308],{"type":29,"value":1309},"accoun",{"type":24,"tag":436,"props":1311,"children":1312},{"style":759},[1313],{"type":29,"value":1314},"t",{"type":24,"tag":436,"props":1316,"children":1317},{"style":753},[1318],{"type":29,"value":1319},">",{"type":24,"tag":436,"props":1321,"children":1322},{"style":449},[1323],{"type":29,"value":1324},".r2.cloudflarestorage.com",{"type":24,"tag":436,"props":1326,"children":1327},{"style":753},[1328],{"type":29,"value":1329}," \u003C",{"type":24,"tag":436,"props":1331,"children":1332},{"style":449},[1333],{"type":29,"value":1334},"ke",{"type":24,"tag":436,"props":1336,"children":1337},{"style":759},[1338],{"type":29,"value":1339},"y",{"type":24,"tag":436,"props":1341,"children":1342},{"style":753},[1343],{"type":29,"value":1319},{"type":24,"tag":436,"props":1345,"children":1346},{"style":753},[1347],{"type":29,"value":1329},{"type":24,"tag":436,"props":1349,"children":1350},{"style":449},[1351],{"type":29,"value":1352},"secre",{"type":24,"tag":436,"props":1354,"children":1355},{"style":759},[1356],{"type":29,"value":1314},{"type":24,"tag":436,"props":1358,"children":1359},{"style":753},[1360],{"type":29,"value":1361},">\n",{"type":24,"tag":25,"props":1363,"children":1364},{},[1365,1367,1372,1374,1380],{"type":29,"value":1366},"A provider that answers ",{"type":24,"tag":32,"props":1368,"children":1370},{"className":1369},[],[1371],{"type":29,"value":319},{"type":29,"value":1373}," while storing nothing is what the table is for. The same principle governs ",{"type":24,"tag":162,"props":1375,"children":1377},{"href":1376},"\u002Fblog\u002Fpresigned-post-enforce-size-content-type",[1378],{"type":29,"value":1379},"presigned POST policies that cap size and content type",{"type":29,"value":1381},": the server enforces what the request declares, so the declaration must be right before the upload starts.",{"type":24,"tag":41,"props":1383,"children":1385},{"id":1384},"how-s3-explorer-fits",[1386],{"type":29,"value":1387},"How S3 Explorer fits",{"type":24,"tag":25,"props":1389,"children":1390},{},[1391,1393,1398,1400,1405,1407,1412,1414,1421],{"type":29,"value":1392},"Which checksum an object carries, and which one the endpoint will return, differ per provider, so the useful check is a read against the real endpoint. S3 Explorer's ",{"type":24,"tag":477,"props":1394,"children":1395},{},[1396],{"type":29,"value":1397},"provider picker",{"type":29,"value":1399}," holds 25 presets across Cloud, Specialist, Self-hosted and Enterprise plus a Custom tile, so an R2 bucket and an S3 bucket open from the same window. Its ",{"type":24,"tag":477,"props":1401,"children":1402},{},[1403],{"type":29,"value":1404},"Metadata",{"type":29,"value":1406}," tab shows the headers the server returned, which is where a stored checksum appears when the endpoint sends one, and ",{"type":24,"tag":477,"props":1408,"children":1409},{},[1410],{"type":29,"value":1411},"test connection",{"type":29,"value":1413}," reports how many buckets the credentials can see. ",{"type":24,"tag":162,"props":1415,"children":1418},{"href":1416,"rel":1417},"https:\u002F\u002Fs3explorer.accessdevops.com",[166],[1419],{"type":29,"value":1420},"S3 Explorer",{"type":29,"value":1422}," runs on Windows, macOS and Linux.",{"type":24,"tag":41,"props":1424,"children":1426},{"id":1425},"checklist",[1427],{"type":29,"value":1428},"Checklist",{"type":24,"tag":48,"props":1430,"children":1433},{"className":1431},[1432],"contains-task-list",[1434,1446,1455,1471,1480,1489,1498],{"type":24,"tag":52,"props":1435,"children":1438},{"className":1436},[1437],"task-list-item",[1439,1444],{"type":24,"tag":1440,"props":1441,"children":1443},"input",{"disabled":790,"type":1442},"checkbox",[],{"type":29,"value":1445}," Every upload path names its checksum algorithm, or documents the SDK default it relies on.",{"type":24,"tag":52,"props":1447,"children":1449},{"className":1448},[1437],[1450,1453],{"type":24,"tag":1440,"props":1451,"children":1452},{"disabled":790,"type":1442},[],{"type":29,"value":1454}," The checksum is computed before the upload and compared with the value the server returns.",{"type":24,"tag":52,"props":1456,"children":1458},{"className":1457},[1437],[1459,1462,1464,1469],{"type":24,"tag":1440,"props":1460,"children":1461},{"disabled":790,"type":1442},[],{"type":29,"value":1463}," Multipart paths that need whole-file verification declare ",{"type":24,"tag":32,"props":1465,"children":1467},{"className":1466},[],[1468],{"type":29,"value":217},{"type":29,"value":1470},", and never compare a composite value to a local CRC.",{"type":24,"tag":52,"props":1472,"children":1474},{"className":1473},[1437],[1475,1478],{"type":24,"tag":1440,"props":1476,"children":1477},{"disabled":790,"type":1442},[],{"type":29,"value":1479}," Downloads enable checksum mode, or the SDK validates the response.",{"type":24,"tag":52,"props":1481,"children":1483},{"className":1482},[1437],[1484,1487],{"type":24,"tag":1440,"props":1485,"children":1486},{"disabled":790,"type":1442},[],{"type":29,"value":1488}," The reader accepts whichever algorithm the object carries, and a missing checksum fails the job.",{"type":24,"tag":52,"props":1490,"children":1492},{"className":1491},[1437],[1493,1496],{"type":24,"tag":1440,"props":1494,"children":1495},{"disabled":790,"type":1442},[],{"type":29,"value":1497}," The CRC implementation is tested against the three published vectors, not only against itself.",{"type":24,"tag":52,"props":1499,"children":1501},{"className":1500},[1437],[1502,1505,1506,1511],{"type":24,"tag":1440,"props":1503,"children":1504},{"disabled":790,"type":1442},[],{"type":29,"value":483},{"type":24,"tag":32,"props":1507,"children":1509},{"className":1508},[],[1510],{"type":29,"value":489},{"type":29,"value":1512}," is capped by the retry policy, not retried as a network error.",{"type":24,"tag":41,"props":1514,"children":1516},{"id":1515},"sources",[1517],{"type":29,"value":1518},"Sources",{"type":24,"tag":48,"props":1520,"children":1521},{},[1522,1568,1600,1626,1644,1664,1697],{"type":24,"tag":52,"props":1523,"children":1524},{},[1525,1531,1533,1539,1540,1545,1546,1551,1553,1559,1561,1567],{"type":24,"tag":162,"props":1526,"children":1528},{"href":164,"rel":1527},[166],[1529],{"type":29,"value":1530},"AWS — default data integrity protections",{"type":29,"value":1532},", 1 December 2024: S3 stores a CRC on new objects even without one from the client; ",{"type":24,"tag":32,"props":1534,"children":1536},{"className":1535},[],[1537],{"type":29,"value":1538},"x-amz-checksum-type",{"type":29,"value":1183},{"type":24,"tag":32,"props":1541,"children":1543},{"className":1542},[],[1544],{"type":29,"value":217},{"type":29,"value":1239},{"type":24,"tag":32,"props":1547,"children":1549},{"className":1548},[],[1550],{"type":29,"value":209},{"type":29,"value":1552},"; the ",{"type":24,"tag":32,"props":1554,"children":1556},{"className":1555},[],[1557],{"type":29,"value":1558},"AuUcyF784aU=",{"type":29,"value":1560}," example for ",{"type":24,"tag":32,"props":1562,"children":1564},{"className":1563},[],[1565],{"type":29,"value":1566},"Hello World!",{"type":29,"value":189},{"type":24,"tag":52,"props":1569,"children":1570},{},[1571,1577,1579,1584,1586,1591,1593,1599],{"type":24,"tag":162,"props":1572,"children":1574},{"href":174,"rel":1573},[166],[1575],{"type":29,"value":1576},"AWS — checking object integrity",{"type":29,"value":1578},": ten algorithms, ",{"type":24,"tag":32,"props":1580,"children":1582},{"className":1581},[],[1583],{"type":29,"value":489},{"type":29,"value":1585}," on a mismatch, ",{"type":24,"tag":32,"props":1587,"children":1589},{"className":1588},[],[1590],{"type":29,"value":217},{"type":29,"value":1592}," limited to CRC algorithms, CRC64NVME when nothing is specified, and REST calls using ",{"type":24,"tag":32,"props":1594,"children":1596},{"className":1595},[],[1597],{"type":29,"value":1598},"x-amz-checksum-\u003Calgorithm>",{"type":29,"value":189},{"type":24,"tag":52,"props":1601,"children":1602},{},[1603,1610,1612,1617,1619,1624],{"type":24,"tag":162,"props":1604,"children":1607},{"href":1605,"rel":1606},"https:\u002F\u002Fdocs.aws.amazon.com\u002Fsdkref\u002Flatest\u002Fguide\u002Ffeature-dataintegrity.html",[166],[1608],{"type":29,"value":1609},"AWS SDK data integrity protections",{"type":29,"value":1611},": per-SDK defaults (CLI v2 CRC64NVME; boto3, Go, Java, JavaScript CRC32) and the ",{"type":24,"tag":32,"props":1613,"children":1615},{"className":1614},[],[1616],{"type":29,"value":840},{"type":29,"value":1618}," \u002F ",{"type":24,"tag":32,"props":1620,"children":1622},{"className":1621},[],[1623],{"type":29,"value":1253},{"type":29,"value":1625}," settings.",{"type":24,"tag":52,"props":1627,"children":1628},{},[1629,1635,1637,1642],{"type":24,"tag":162,"props":1630,"children":1632},{"href":494,"rel":1631},[166],[1633],{"type":29,"value":1634},"S3 error catalogue",{"type":29,"value":1636},": ",{"type":24,"tag":32,"props":1638,"children":1640},{"className":1639},[],[1641],{"type":29,"value":489},{"type":29,"value":1643},", 400, \"The Content-MD5 you specified did not match what we received.\"",{"type":24,"tag":52,"props":1645,"children":1646},{},[1647,1654,1655,1662],{"type":24,"tag":162,"props":1648,"children":1651},{"href":1649,"rel":1650},"https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonS3\u002Flatest\u002FAPI\u002FAPI_GetObject.html",[166],[1652],{"type":29,"value":1653},"GetObject",{"type":29,"value":1206},{"type":24,"tag":162,"props":1656,"children":1659},{"href":1657,"rel":1658},"https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonS3\u002Flatest\u002FAPI\u002FAPI_HeadObject.html",[166],[1660],{"type":29,"value":1661},"HeadObject",{"type":29,"value":1663},": \"To retrieve the checksum, this mode must be enabled\".",{"type":24,"tag":52,"props":1665,"children":1666},{},[1667,1673,1675,1680,1682,1687,1689,1695],{"type":24,"tag":162,"props":1668,"children":1670},{"href":659,"rel":1669},[166],[1671],{"type":29,"value":1672},"R2 S3 compatibility",{"type":29,"value":1674},": CRC64NVME ",{"type":24,"tag":32,"props":1676,"children":1678},{"className":1677},[],[1679],{"type":29,"value":217},{"type":29,"value":1681}," only, CRC32 ",{"type":24,"tag":32,"props":1683,"children":1685},{"className":1684},[],[1686],{"type":29,"value":209},{"type":29,"value":1688}," only; ",{"type":24,"tag":162,"props":1690,"children":1692},{"href":668,"rel":1691},[166],[1693],{"type":29,"value":1694},"release notes",{"type":29,"value":1696},", 3 July 2025.",{"type":24,"tag":52,"props":1698,"children":1699},{},[1700,1702,1707,1709,1715,1717,1723],{"type":29,"value":1701},"Measured 9 November 2026 against SeaweedFS 4.50 with ",{"type":24,"tag":32,"props":1703,"children":1705},{"className":1704},[],[1706],{"type":29,"value":251},{"type":29,"value":1708}," 1.43.126 and ",{"type":24,"tag":32,"props":1710,"children":1712},{"className":1711},[],[1713],{"type":29,"value":1714},"awscrt",{"type":29,"value":1716},"; the probe is ",{"type":24,"tag":32,"props":1718,"children":1720},{"className":1719},[],[1721],{"type":29,"value":1722},"scripts\u002Fchecksum-probe.py",{"type":29,"value":189},{"type":24,"tag":1725,"props":1726,"children":1727},"style",{},[1728],{"type":29,"value":1729},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":7,"searchDepth":617,"depth":617,"links":1731},[1732,1733,1734,1735,1736,1737,1738],{"id":43,"depth":617,"text":46},{"id":102,"depth":617,"text":105},{"id":229,"depth":617,"text":232},{"id":725,"depth":617,"text":728},{"id":1384,"depth":617,"text":1387},{"id":1425,"depth":617,"text":1428},{"id":1515,"depth":617,"text":1518},"markdown","content:blog:crc64nvme-new-integrity-default.md","content","blog\u002Fcrc64nvme-new-integrity-default.md","blog\u002Fcrc64nvme-new-integrity-default","md",[1746,1754,1761,1767,1769,1775,1781,1785,1791,1796,1801,1809,1816,1822],{"_path":1747,"title":1748,"description":1749,"tags":1750},"\u002Fblog\u002Fabac-object-tags-vs-prefix-policies","Object tags or prefixes: which access model survives growth","Prefixes decide what a principal can list, tags decide what it can read. Two AWS limits break tag-only designs, and prefix policies run out at about 60 tenants.",[13,1751,1752,1753],"iam","governance","access-control",{"_path":1755,"title":1756,"description":1757,"tags":1758},"\u002Fblog\u002Fassumerole-chaining-one-hour-ceiling","Chained AssumeRole sessions stop at one hour, silently","A role that assumes another role gets 3600 seconds, whatever its MaxSessionDuration allows. The exact error, and the batch job that dies at minute 61.",[13,1751,1759,1760],"sts","aws",{"_path":1762,"title":1763,"description":1764,"tags":1765},"\u002Fblog\u002Fbucket-policy-allow-still-denied","When two policies allow and the request still fails","S3 returns AccessDenied even when the bucket and IAM policies both allow. Here is the five-gate evaluation order and where the hidden deny sits.",[13,1751,1766,1753],"security",{"_path":4,"title":8,"description":9,"tags":1768},[13,14,15],{"_path":1770,"title":1771,"description":1772,"tags":1773},"\u002Fblog\u002Flist-pagination-1000-pages","S3 LIST returns 1,000 entries a page, folders included","One ListObjectsV2 response holds at most 1,000 entries, and a folder rollup spends one of them. Measured on two endpoints, plus the loop that stops too early.",[13,15,1774],"performance",{"_path":521,"title":1776,"description":1777,"tags":1778},"Multipart upload: the 5 MiB floor and the 10,000-part ceiling","A single PUT stops at 5 GiB. Splitting a 60 GB file moves the failure rather than removing it: the 5 MiB part floor is checked last, after every byte is sent.",[13,1779,1780],"uploads","multipart",{"_path":1376,"title":1782,"description":1783,"tags":1784},"Presigned POST: cap size and type before the upload starts","A presigned PUT signs a URL, not a byte count. A POST policy adds a size range and a content-type condition that S3 enforces while the body arrives.",[13,1766,1779],{"_path":1786,"title":1787,"description":1788,"tags":1789},"\u002Fblog\u002Fpresigned-url-leakage","Presigned URLs leak into logs, referrers and chat","A presigned URL is a working credential stored as text. Where the copy ends up, what it still grants after redaction, and the 15-minute fix.",[13,1766,1790],"presigned-urls",{"_path":1792,"title":1793,"description":1794,"tags":1795},"\u002Fblog\u002Fpresigned-url-seven-day-ceiling","Presigned URLs expire at seven days. Then what?","Seven days is not a setting: it is how long the SigV4 signing key lives. What happens at that wall, and the three patterns that outlive it.",[13,1766,1790],{"_path":1797,"title":1798,"description":1799,"tags":1800},"\u002Fblog\u002Fresume-interrupted-multipart-upload","Resuming a multipart upload you never meant to abandon","An upload ID never expires, so a crashed upload is resumable without the state file — measured: the part list is the real state, and it can lie about the ETag.",[13,1779,1780],{"_path":1802,"title":1803,"description":1804,"tags":1805},"\u002Fblog\u002Fs3-explorer-walkthrough","S3 Explorer walkthrough: zero to browsing in 5 minutes","A step-by-step walkthrough of S3 Explorer against a live S3 endpoint: pick a provider, test, browse buckets, inspect metadata, read request metrics.",[13,1806,1807,1808],"tutorial","tooling","seaweedfs",{"_path":1810,"title":1811,"description":1812,"tags":1813},"\u002Fblog\u002Fs3-in-2027-egress-bill-changes","S3 in 2027: the egress bill changes on 12 January","The EU Data Act removes switching fees in January 2027 — but not everyday egress. The five things you need to know about your buckets before that date.",[13,1814,1752,1815],"pricing","cloud",{"_path":1817,"title":1818,"description":1819,"tags":1820},"\u002Fblog\u002Fs3-total-cost-of-ownership","The real cost of S3: 8 providers compared, egress included","Storage is 20% of your bill. We modelled 8 S3 providers across 4 workloads — backup, app, media, DR — with storage, egress, requests and replication.",[13,1814,1815,1821],"cost-optimisation",{"_path":1823,"title":1824,"description":1825,"tags":1826},"\u002Fblog\u002Fwelcome","Why we write about S3, and how this blog is made","Object storage is everywhere and most advice about it is stale. What this blog covers, how each article is sourced, and what we measure to know it helped.",[13,1815],1794568659400]