[{"data":1,"prerenderedAt":2199},["ShallowReactive",2],{"post-\u002Fblog\u002Fmultipart-5mb-floor-10000-parts":3,"blog-index":2140},{"_path":4,"_dir":5,"_draft":6,"_partial":6,"_locale":7,"title":8,"description":9,"date":10,"author":11,"tags":12,"guide":16,"related":17,"body":20,"_type":2134,"_id":2135,"_source":2136,"_file":2137,"_stem":2138,"_extension":2139},"\u002Fblog\u002Fmultipart-5mb-floor-10000-parts","blog",false,"","Multipart upload: the 5 MiB floor and the 10,000-part ceiling","A single PUT stops at 5 GiB. Splitting a 60 GB file moves the failure rather than removing it: the 5 MiB part floor is checked last, after every byte is sent.","2026-11-06","Clement Baranger",[13,14,15],"s3","uploads","multipart","incomplete-multipart-uploads",[18,19],"presigned-post-enforce-size-content-type","s3-total-cost-of-ownership",{"type":21,"children":22,"toc":2125},"root",[23,40,47,109,115,127,132,141,146,182,361,394,407,413,449,545,573,598,608,618,644,662,696,714,724,746,752,760,1048,1053,1063,1270,1286,1537,1547,1574,1584,1770,1776,1827,1833,1942,1948,2119],{"type":24,"tag":25,"props":26,"children":27},"element","p",{},[28,31,38],{"type":29,"value":30},"text","A 60 GB backup fails with a single ",{"type":24,"tag":32,"props":33,"children":35},"code",{"className":34},[],[36],{"type":29,"value":37},"PUT",{"type":29,"value":39},": the request never completes, and minutes later the client reports a dropped connection instead of a reason. Splitting the file into parts is the fix — and the moment multipart's own two walls appear, a 5 MiB minimum per part and a hard count of 10,000 parts. One is checked before the first byte, the other after the last.",{"type":24,"tag":41,"props":42,"children":44},"h2",{"id":43},"the-short-version",[45],{"type":29,"value":46},"The short version",{"type":24,"tag":48,"props":49,"children":50},"ul",{},[51,57,62,67,104],{"type":24,"tag":52,"props":53,"children":54},"li",{},[55],{"type":29,"value":56},"A single upload request cannot carry more than 5 GiB. Anything bigger is split into parts first, by the SDK or by you.",{"type":24,"tag":52,"props":58,"children":59},{},[60],{"type":29,"value":61},"Multipart allows each part between 5 MiB and 5 GiB, at most 10,000 parts per object, and no minimum on the last part.",{"type":24,"tag":52,"props":63,"children":64},{},[65],{"type":29,"value":66},"Part size × 10,000 must cover the object. Multiply the 5 GiB maximum part by the parts ceiling and you get 48.8 TiB, the largest object the protocol can hold.",{"type":24,"tag":52,"props":68,"children":69},{},[70,72,78,80,86,88,94,96,102],{"type":29,"value":71},"The 5 MiB floor is enforced at the ",{"type":24,"tag":73,"props":74,"children":75},"strong",{},[76],{"type":29,"value":77},"end",{"type":29,"value":79},": all parts upload with ",{"type":24,"tag":32,"props":81,"children":83},{"className":82},[],[84],{"type":29,"value":85},"200",{"type":29,"value":87},", then ",{"type":24,"tag":32,"props":89,"children":91},{"className":90},[],[92],{"type":29,"value":93},"CompleteMultipartUpload",{"type":29,"value":95}," answers ",{"type":24,"tag":32,"props":97,"children":99},{"className":98},[],[100],{"type":29,"value":101},"400 EntityTooSmall",{"type":29,"value":103}," and the object never exists. Measured below.",{"type":24,"tag":52,"props":105,"children":106},{},[107],{"type":29,"value":108},"The damage is bandwidth and orphaned parts, not lost files: a rejected completion leaves every part behind, billed as storage until an abort removes them.",{"type":24,"tag":41,"props":110,"children":112},{"id":111},"what-actually-happens",[113],{"type":29,"value":114},"What actually happens",{"type":24,"tag":25,"props":116,"children":117},{},[118,120,125],{"type":29,"value":119},"Amazon S3 documents a single ",{"type":24,"tag":32,"props":121,"children":123},{"className":122},[],[124],{"type":29,"value":37},{"type":29,"value":126}," as carrying \"up to 5 GB\" — 5,368,709,120 bytes, so GiB in practice — and multipart as the path for anything larger. The limits page adds three numbers: part numbers run from 1 to 10,000 inclusive, part size runs from 5 MiB to 5 GiB, and \"there is no minimum size limit on the last part of your multipart upload\".",{"type":24,"tag":25,"props":128,"children":129},{},[130],{"type":29,"value":131},"That page gives the maximum object size as 48.8 TiB, while the upload guide promises \"up to 50 TB\". Both are the same ceiling in different units: 10,000 × 5,368,709,120 bytes is 53,687,091,200,000 bytes, or 48.83 TiB, or 53.69 TB. The maximum object size is not an independent rule. It is the parts ceiling multiplied by the maximum part, which is why raising one without the other changes nothing.",{"type":24,"tag":25,"props":133,"children":134},{},[135],{"type":24,"tag":136,"props":137,"children":140},"img",{"alt":138,"src":139},"Four steps of a multipart upload — initiate, upload parts, complete the object — with the part count checked at upload time and the 5 MiB floor checked only at completion, so a too-small part is refused after every byte has already been sent","\u002Fdiagrams\u002Fmultipart-5mb-floor-10000-parts.svg",[],{"type":24,"tag":25,"props":142,"children":143},{},[144],{"type":29,"value":145},"The part size is therefore a calculation, not a setting: divide the object size by 10,000 and round up to at least 5 MiB. Below that, the upload runs out of part numbers before it runs out of bytes.",{"type":24,"tag":25,"props":147,"children":148},{},[149,151,157,159,165,167,173,175,180],{"type":29,"value":150},"What clients do with it is more interesting than the documentation. ",{"type":24,"tag":32,"props":152,"children":154},{"className":153},[],[155],{"type":29,"value":156},"s3transfer",{"type":29,"value":158}," 0.19.3 — the layer behind ",{"type":24,"tag":32,"props":160,"children":162},{"className":161},[],[163],{"type":29,"value":164},"boto3.upload_file",{"type":29,"value":166}," — ships ",{"type":24,"tag":32,"props":168,"children":170},{"className":169},[],[171],{"type":29,"value":172},"ChunksizeAdjuster",{"type":29,"value":174},", which starts from the configured 8 MiB and ",{"type":24,"tag":73,"props":176,"children":177},{},[178],{"type":29,"value":179},"doubles",{"type":29,"value":181}," the chunk size until the part count fits under 10,000, then clamps the result to the 5 MiB floor and the 5 GiB ceiling. Called here through the library's own function, so these are the client's numbers, not mine:",{"type":24,"tag":183,"props":184,"children":185},"table",{},[186,220],{"type":24,"tag":187,"props":188,"children":189},"thead",{},[190],{"type":24,"tag":191,"props":192,"children":193},"tr",{},[194,200,205,210,215],{"type":24,"tag":195,"props":196,"children":197},"th",{},[198],{"type":29,"value":199},"Object",{"type":24,"tag":195,"props":201,"children":202},{},[203],{"type":29,"value":204},"Minimum part to fit 10,000",{"type":24,"tag":195,"props":206,"children":207},{},[208],{"type":29,"value":209},"Part size the SDK picks",{"type":24,"tag":195,"props":211,"children":212},{},[213],{"type":29,"value":214},"Parts",{"type":24,"tag":195,"props":216,"children":217},{},[218],{"type":29,"value":219},"Last part",{"type":24,"tag":221,"props":222,"children":223},"tbody",{},[224,252,279,306,333],{"type":24,"tag":191,"props":225,"children":226},{},[227,233,238,243,248],{"type":24,"tag":228,"props":229,"children":230},"td",{},[231],{"type":29,"value":232},"60 GiB",{"type":24,"tag":228,"props":234,"children":235},{},[236],{"type":29,"value":237},"6,442,451 B (6.1 MiB)",{"type":24,"tag":228,"props":239,"children":240},{},[241],{"type":29,"value":242},"8 MiB",{"type":24,"tag":228,"props":244,"children":245},{},[246],{"type":29,"value":247},"7,680",{"type":24,"tag":228,"props":249,"children":250},{},[251],{"type":29,"value":242},{"type":24,"tag":191,"props":253,"children":254},{},[255,260,265,270,275],{"type":24,"tag":228,"props":256,"children":257},{},[258],{"type":29,"value":259},"500 GiB",{"type":24,"tag":228,"props":261,"children":262},{},[263],{"type":29,"value":264},"53,687,092 B (51.2 MiB)",{"type":24,"tag":228,"props":266,"children":267},{},[268],{"type":29,"value":269},"64 MiB",{"type":24,"tag":228,"props":271,"children":272},{},[273],{"type":29,"value":274},"8,000",{"type":24,"tag":228,"props":276,"children":277},{},[278],{"type":29,"value":269},{"type":24,"tag":191,"props":280,"children":281},{},[282,287,292,297,302],{"type":24,"tag":228,"props":283,"children":284},{},[285],{"type":29,"value":286},"1 TiB",{"type":24,"tag":228,"props":288,"children":289},{},[290],{"type":29,"value":291},"109,951,163 B (104.9 MiB)",{"type":24,"tag":228,"props":293,"children":294},{},[295],{"type":29,"value":296},"128 MiB",{"type":24,"tag":228,"props":298,"children":299},{},[300],{"type":29,"value":301},"8,192",{"type":24,"tag":228,"props":303,"children":304},{},[305],{"type":29,"value":296},{"type":24,"tag":191,"props":307,"children":308},{},[309,314,319,324,329],{"type":24,"tag":228,"props":310,"children":311},{},[312],{"type":29,"value":313},"5 TiB",{"type":24,"tag":228,"props":315,"children":316},{},[317],{"type":29,"value":318},"549,755,814 B (524.3 MiB)",{"type":24,"tag":228,"props":320,"children":321},{},[322],{"type":29,"value":323},"1 GiB",{"type":24,"tag":228,"props":325,"children":326},{},[327],{"type":29,"value":328},"5,120",{"type":24,"tag":228,"props":330,"children":331},{},[332],{"type":29,"value":323},{"type":24,"tag":191,"props":334,"children":335},{},[336,341,346,351,356],{"type":24,"tag":228,"props":337,"children":338},{},[339],{"type":29,"value":340},"48.8 TiB",{"type":24,"tag":228,"props":342,"children":343},{},[344],{"type":29,"value":345},"5,365,616,744 B (5.0 GiB)",{"type":24,"tag":228,"props":347,"children":348},{},[349],{"type":29,"value":350},"5 GiB",{"type":24,"tag":228,"props":352,"children":353},{},[354],{"type":29,"value":355},"9,995",{"type":24,"tag":228,"props":357,"children":358},{},[359],{"type":29,"value":360},"1.2 GiB",{"type":24,"tag":362,"props":363,"children":367},"pre",{"className":364,"code":365,"language":366,"meta":7,"style":7},"language-bash shiki shiki-themes github-light-high-contrast github-dark-high-contrast","python3 scripts\u002Fmultipart-probe.py plan\n","bash",[368],{"type":24,"tag":32,"props":369,"children":370},{"__ignoreMap":7},[371],{"type":24,"tag":372,"props":373,"children":376},"span",{"class":374,"line":375},"line",1,[377,383,389],{"type":24,"tag":372,"props":378,"children":380},{"style":379},"--shiki-default:#702C00;--shiki-dark:#FFB757",[381],{"type":29,"value":382},"python3",{"type":24,"tag":372,"props":384,"children":386},{"style":385},"--shiki-default:#032563;--shiki-dark:#ADDCFF",[387],{"type":29,"value":388}," scripts\u002Fmultipart-probe.py",{"type":24,"tag":372,"props":390,"children":391},{"style":385},[392],{"type":29,"value":393}," plan\n",{"type":24,"tag":25,"props":395,"children":396},{},[397,399,405],{"type":29,"value":398},"The doubling matters. A 5 TiB object needs 524 MiB parts; the library ships 1 GiB parts, twice the necessary size. When a part fails, the retry re-sends 1 GiB instead of 524 MiB — at 100 Mbit\u002Fs, about 90 seconds, paid again on every retry. For very large objects, pass ",{"type":24,"tag":32,"props":400,"children":402},{"className":401},[],[403],{"type":29,"value":404},"multipart_chunksize",{"type":29,"value":406}," explicitly.",{"type":24,"tag":41,"props":408,"children":410},{"id":409},"where-it-bites",[411],{"type":29,"value":412},"Where it bites",{"type":24,"tag":25,"props":414,"children":415},{},[416,418,424,426,432,434,440,442,447],{"type":29,"value":417},"I ran four uploads against SeaweedFS 4.50 — ",{"type":24,"tag":32,"props":419,"children":421},{"className":420},[],[422],{"type":29,"value":423},"weed server -s3",{"type":29,"value":425},", a container on the same machine at ",{"type":24,"tag":32,"props":427,"children":429},{"className":428},[],[430],{"type":29,"value":431},"http:\u002F\u002F127.0.0.1:18333",{"type":29,"value":433}," — with ",{"type":24,"tag":32,"props":435,"children":437},{"className":436},[],[438],{"type":29,"value":439},"boto3",{"type":29,"value":441}," 1.43.125. Every part uploaded with a ",{"type":24,"tag":32,"props":443,"children":445},{"className":444},[],[446],{"type":29,"value":85},{"type":29,"value":448},". Then:",{"type":24,"tag":183,"props":450,"children":451},{},[452,468],{"type":24,"tag":187,"props":453,"children":454},{},[455],{"type":24,"tag":191,"props":456,"children":457},{},[458,463],{"type":24,"tag":195,"props":459,"children":460},{},[461],{"type":29,"value":462},"Parts sent",{"type":24,"tag":195,"props":464,"children":465},{},[466],{"type":29,"value":467},"Result",{"type":24,"tag":221,"props":469,"children":470},{},[471,489,507,526],{"type":24,"tag":191,"props":472,"children":473},{},[474,479],{"type":24,"tag":228,"props":475,"children":476},{},[477],{"type":29,"value":478},"1 MiB, 1 MiB, 1 MiB, 1 KiB",{"type":24,"tag":228,"props":480,"children":481},{},[482,487],{"type":24,"tag":32,"props":483,"children":485},{"className":484},[],[486],{"type":29,"value":101},{"type":29,"value":488}," — \"Your proposed upload is smaller than the minimum allowed object size.\"",{"type":24,"tag":191,"props":490,"children":491},{},[492,497],{"type":24,"tag":228,"props":493,"children":494},{},[495],{"type":29,"value":496},"5 MiB − 1 byte, 1 MiB",{"type":24,"tag":228,"props":498,"children":499},{},[500,505],{"type":24,"tag":32,"props":501,"children":503},{"className":502},[],[504],{"type":29,"value":101},{"type":29,"value":506}," — same message",{"type":24,"tag":191,"props":508,"children":509},{},[510,515],{"type":24,"tag":228,"props":511,"children":512},{},[513],{"type":29,"value":514},"5 MiB, 5 MiB, 1 KiB",{"type":24,"tag":228,"props":516,"children":517},{},[518,520],{"type":29,"value":519},"object stored, 10,486,784 bytes, ETag ",{"type":24,"tag":32,"props":521,"children":523},{"className":522},[],[524],{"type":29,"value":525},"\"fd20d710…-3\"",{"type":24,"tag":191,"props":527,"children":528},{},[529,534],{"type":24,"tag":228,"props":530,"children":531},{},[532],{"type":29,"value":533},"1 KiB, single part",{"type":24,"tag":228,"props":535,"children":536},{},[537,539],{"type":29,"value":538},"object stored, 1,024 bytes, ETag ",{"type":24,"tag":32,"props":540,"children":542},{"className":541},[],[543],{"type":29,"value":544},"\"64a3b22c…-1\"",{"type":24,"tag":362,"props":546,"children":548},{"className":364,"code":547,"language":366,"meta":7,"style":7},"python3 scripts\u002Fmultipart-probe.py probe http:\u002F\u002F127.0.0.1:18333\n",[549],{"type":24,"tag":32,"props":550,"children":551},{"__ignoreMap":7},[552],{"type":24,"tag":372,"props":553,"children":554},{"class":374,"line":375},[555,559,563,568],{"type":24,"tag":372,"props":556,"children":557},{"style":379},[558],{"type":29,"value":382},{"type":24,"tag":372,"props":560,"children":561},{"style":385},[562],{"type":29,"value":388},{"type":24,"tag":372,"props":564,"children":565},{"style":385},[566],{"type":29,"value":567}," probe",{"type":24,"tag":372,"props":569,"children":570},{"style":385},[571],{"type":29,"value":572}," http:\u002F\u002F127.0.0.1:18333\n",{"type":24,"tag":25,"props":574,"children":575},{},[576,581,583,589,591,596],{"type":24,"tag":73,"props":577,"children":578},{},[579],{"type":29,"value":580},"The floor is a completion-time check.",{"type":29,"value":582}," In the first two rows every ",{"type":24,"tag":32,"props":584,"children":586},{"className":585},[],[587],{"type":29,"value":588},"UploadPart",{"type":29,"value":590}," succeeded; the rejection came on ",{"type":24,"tag":32,"props":592,"children":594},{"className":593},[],[595],{"type":29,"value":93},{"type":29,"value":597},", after the whole payload had crossed the network. A client that retries re-sends the same wrong part size and fails identically. The upload looks healthy until the last call.",{"type":24,"tag":25,"props":599,"children":600},{},[601,606],{"type":24,"tag":73,"props":602,"children":603},{},[604],{"type":29,"value":605},"It is a floor, exactly.",{"type":29,"value":607}," 5,242,880 bytes passes, 5,242,879 does not. Code that derives a part size by integer division gets it wrong on the remainder.",{"type":24,"tag":25,"props":609,"children":610},{},[611,616],{"type":24,"tag":73,"props":612,"children":613},{},[614],{"type":29,"value":615},"The last part is exempt, and must be.",{"type":29,"value":617}," A 10,486,784-byte object finished with a 1,024-byte part. Code that pads the final part to 5 MiB uploads megabytes of nothing and corrupts the object's size.",{"type":24,"tag":25,"props":619,"children":620},{},[621,626,628,634,636,642],{"type":24,"tag":73,"props":622,"children":623},{},[624],{"type":29,"value":625},"The count is checked at a different moment, with a different code.",{"type":29,"value":627}," Parts 9,999 and 10,000 were accepted; 10,001 answered ",{"type":24,"tag":32,"props":629,"children":631},{"className":630},[],[632],{"type":29,"value":633},"400 InvalidArgument",{"type":29,"value":635}," — during the upload, not at completion, so a client that treats every 400 the same reports the wrong cause. The wording is not portable: SeaweedFS returns the terse ",{"type":24,"tag":32,"props":637,"children":639},{"className":638},[],[640],{"type":29,"value":641},"Invalid Argument",{"type":29,"value":643},", moto 5.2.5 returns \"Part number must be an integer between 1 and 10000, inclusive\". Catch on the code, log the message.",{"type":24,"tag":25,"props":645,"children":646},{},[647,652,654,660],{"type":24,"tag":73,"props":648,"children":649},{},[650],{"type":29,"value":651},"A failed completion leaves the parts behind.",{"type":29,"value":653}," Nothing deletes them until you write a lifecycle rule, and they are billed as storage at the object's size. An upload that dies at 90% of 5 TiB leaves about 4.5 TiB of orphaned parts — a line on an invoice rather than in a log. Abort in a ",{"type":24,"tag":32,"props":655,"children":657},{"className":656},[],[658],{"type":29,"value":659},"finally",{"type":29,"value":661}," block.",{"type":24,"tag":25,"props":663,"children":664},{},[665,670,672,678,680,686,688,694],{"type":24,"tag":73,"props":666,"children":667},{},[668],{"type":29,"value":669},"The ETag changes shape.",{"type":29,"value":671}," The stored value was ",{"type":24,"tag":32,"props":673,"children":675},{"className":674},[],[676],{"type":29,"value":677},"\"fd20d710cec39419f74133c915a37638-3\"",{"type":29,"value":679},": the MD5 of the concatenated part MD5s with the part count appended. The file's own MD5 is ",{"type":24,"tag":32,"props":681,"children":683},{"className":682},[],[684],{"type":29,"value":685},"a96f20bf…",{"type":29,"value":687},", so a ",{"type":24,"tag":32,"props":689,"children":691},{"className":690},[],[692],{"type":29,"value":693},"Content-MD5",{"type":29,"value":695}," comparison against the ETag reports corruption on a perfect upload.",{"type":24,"tag":25,"props":697,"children":698},{},[699,704,706,712],{"type":24,"tag":73,"props":700,"children":701},{},[702],{"type":29,"value":703},"Provider ceilings differ.",{"type":29,"value":705}," Cloudflare R2 documents multipart objects to 5 TiB with 10,000 parts and parts of 5 MiB to 5 GiB; AWS documents 48.8 TiB. Same protocol, a factor of ten apart — a pipeline sized against one endpoint hits a ",{"type":24,"tag":32,"props":707,"children":709},{"className":708},[],[710],{"type":29,"value":711},"400",{"type":29,"value":713}," on the other, and the message does not say which limit you met.",{"type":24,"tag":25,"props":715,"children":716},{},[717,722],{"type":24,"tag":73,"props":718,"children":719},{},[720],{"type":29,"value":721},"Sizing by the last successful upload is how this goes wrong.",{"type":29,"value":723}," A 100 GiB job works with 8 MiB parts; point it at 5 TiB and the ceiling appears at part 10,001, hours in. At 8 MiB parts, 10,000 × 8 MiB = 78.1 GiB is the largest object that default ever finishes.",{"type":24,"tag":25,"props":725,"children":726},{},[727,729,736,738,744],{"type":29,"value":728},"Orphaned parts are what ",{"type":24,"tag":730,"props":731,"children":733},"a",{"href":732},"\u002Fblog\u002Fs3-total-cost-of-ownership",[734],{"type":29,"value":735},"the rest of an S3 bill is made of",{"type":29,"value":737},"; the reverse problem, capping a single upload before it starts, is covered in ",{"type":24,"tag":730,"props":739,"children":741},{"href":740},"\u002Fblog\u002Fpresigned-post-enforce-size-content-type",[742],{"type":29,"value":743},"presigned POST size limits",{"type":29,"value":745},".",{"type":24,"tag":41,"props":747,"children":749},{"id":748},"doing-it-right",[750],{"type":29,"value":751},"Doing it right",{"type":24,"tag":25,"props":753,"children":754},{},[755],{"type":24,"tag":73,"props":756,"children":757},{},[758],{"type":29,"value":759},"1. Compute the part size from the object, not from a constant.",{"type":24,"tag":362,"props":761,"children":765},{"className":762,"code":763,"language":764,"meta":7,"style":7},"language-python shiki shiki-themes github-light-high-contrast github-dark-high-contrast","MIB, GIB = 1024**2, 1024**3\nMAX_PARTS, MIN_PART, MAX_PART = 10_000, 5 * MIB, 5 * GIB\n\ndef part_size(object_size: int, target_parts: int = 5_000) -> int:\n    \"\"\"Big enough to fit in 10,000 parts, small enough to retry.\"\"\"\n    return max(MIN_PART, min(MAX_PART, -(-object_size \u002F\u002F target_parts)))\n","python",[766],{"type":24,"tag":32,"props":767,"children":768},{"__ignoreMap":7},[769,828,900,910,967,976],{"type":24,"tag":372,"props":770,"children":771},{"class":374,"line":375},[772,778,784,789,795,800,805,810,814,819,823],{"type":24,"tag":372,"props":773,"children":775},{"style":774},"--shiki-default:#023B95;--shiki-dark:#91CBFF",[776],{"type":29,"value":777},"MIB",{"type":24,"tag":372,"props":779,"children":781},{"style":780},"--shiki-default:#0E1116;--shiki-dark:#F0F3F6",[782],{"type":29,"value":783},", ",{"type":24,"tag":372,"props":785,"children":786},{"style":774},[787],{"type":29,"value":788},"GIB",{"type":24,"tag":372,"props":790,"children":792},{"style":791},"--shiki-default:#A0111F;--shiki-dark:#FF9492",[793],{"type":29,"value":794}," =",{"type":24,"tag":372,"props":796,"children":797},{"style":774},[798],{"type":29,"value":799}," 1024",{"type":24,"tag":372,"props":801,"children":802},{"style":791},[803],{"type":29,"value":804},"**",{"type":24,"tag":372,"props":806,"children":807},{"style":774},[808],{"type":29,"value":809},"2",{"type":24,"tag":372,"props":811,"children":812},{"style":780},[813],{"type":29,"value":783},{"type":24,"tag":372,"props":815,"children":816},{"style":774},[817],{"type":29,"value":818},"1024",{"type":24,"tag":372,"props":820,"children":821},{"style":791},[822],{"type":29,"value":804},{"type":24,"tag":372,"props":824,"children":825},{"style":774},[826],{"type":29,"value":827},"3\n",{"type":24,"tag":372,"props":829,"children":831},{"class":374,"line":830},2,[832,837,841,846,850,855,859,864,868,873,878,883,887,891,895],{"type":24,"tag":372,"props":833,"children":834},{"style":774},[835],{"type":29,"value":836},"MAX_PARTS",{"type":24,"tag":372,"props":838,"children":839},{"style":780},[840],{"type":29,"value":783},{"type":24,"tag":372,"props":842,"children":843},{"style":774},[844],{"type":29,"value":845},"MIN_PART",{"type":24,"tag":372,"props":847,"children":848},{"style":780},[849],{"type":29,"value":783},{"type":24,"tag":372,"props":851,"children":852},{"style":774},[853],{"type":29,"value":854},"MAX_PART",{"type":24,"tag":372,"props":856,"children":857},{"style":791},[858],{"type":29,"value":794},{"type":24,"tag":372,"props":860,"children":861},{"style":774},[862],{"type":29,"value":863}," 10_000",{"type":24,"tag":372,"props":865,"children":866},{"style":780},[867],{"type":29,"value":783},{"type":24,"tag":372,"props":869,"children":870},{"style":774},[871],{"type":29,"value":872},"5",{"type":24,"tag":372,"props":874,"children":875},{"style":791},[876],{"type":29,"value":877}," *",{"type":24,"tag":372,"props":879,"children":880},{"style":774},[881],{"type":29,"value":882}," MIB",{"type":24,"tag":372,"props":884,"children":885},{"style":780},[886],{"type":29,"value":783},{"type":24,"tag":372,"props":888,"children":889},{"style":774},[890],{"type":29,"value":872},{"type":24,"tag":372,"props":892,"children":893},{"style":791},[894],{"type":29,"value":877},{"type":24,"tag":372,"props":896,"children":897},{"style":774},[898],{"type":29,"value":899}," GIB\n",{"type":24,"tag":372,"props":901,"children":903},{"class":374,"line":902},3,[904],{"type":24,"tag":372,"props":905,"children":907},{"emptyLinePlaceholder":906},true,[908],{"type":29,"value":909},"\n",{"type":24,"tag":372,"props":911,"children":913},{"class":374,"line":912},4,[914,919,925,930,935,940,944,948,953,958,962],{"type":24,"tag":372,"props":915,"children":916},{"style":791},[917],{"type":29,"value":918},"def",{"type":24,"tag":372,"props":920,"children":922},{"style":921},"--shiki-default:#622CBC;--shiki-dark:#DBB7FF",[923],{"type":29,"value":924}," part_size",{"type":24,"tag":372,"props":926,"children":927},{"style":780},[928],{"type":29,"value":929},"(object_size: ",{"type":24,"tag":372,"props":931,"children":932},{"style":774},[933],{"type":29,"value":934},"int",{"type":24,"tag":372,"props":936,"children":937},{"style":780},[938],{"type":29,"value":939},", target_parts: ",{"type":24,"tag":372,"props":941,"children":942},{"style":774},[943],{"type":29,"value":934},{"type":24,"tag":372,"props":945,"children":946},{"style":791},[947],{"type":29,"value":794},{"type":24,"tag":372,"props":949,"children":950},{"style":774},[951],{"type":29,"value":952}," 5_000",{"type":24,"tag":372,"props":954,"children":955},{"style":780},[956],{"type":29,"value":957},") -> ",{"type":24,"tag":372,"props":959,"children":960},{"style":774},[961],{"type":29,"value":934},{"type":24,"tag":372,"props":963,"children":964},{"style":780},[965],{"type":29,"value":966},":\n",{"type":24,"tag":372,"props":968,"children":970},{"class":374,"line":969},5,[971],{"type":24,"tag":372,"props":972,"children":973},{"style":385},[974],{"type":29,"value":975},"    \"\"\"Big enough to fit in 10,000 parts, small enough to retry.\"\"\"\n",{"type":24,"tag":372,"props":977,"children":979},{"class":374,"line":978},6,[980,985,990,995,999,1003,1008,1012,1016,1020,1025,1029,1033,1038,1043],{"type":24,"tag":372,"props":981,"children":982},{"style":791},[983],{"type":29,"value":984},"    return",{"type":24,"tag":372,"props":986,"children":987},{"style":774},[988],{"type":29,"value":989}," max",{"type":24,"tag":372,"props":991,"children":992},{"style":780},[993],{"type":29,"value":994},"(",{"type":24,"tag":372,"props":996,"children":997},{"style":774},[998],{"type":29,"value":845},{"type":24,"tag":372,"props":1000,"children":1001},{"style":780},[1002],{"type":29,"value":783},{"type":24,"tag":372,"props":1004,"children":1005},{"style":774},[1006],{"type":29,"value":1007},"min",{"type":24,"tag":372,"props":1009,"children":1010},{"style":780},[1011],{"type":29,"value":994},{"type":24,"tag":372,"props":1013,"children":1014},{"style":774},[1015],{"type":29,"value":854},{"type":24,"tag":372,"props":1017,"children":1018},{"style":780},[1019],{"type":29,"value":783},{"type":24,"tag":372,"props":1021,"children":1022},{"style":791},[1023],{"type":29,"value":1024},"-",{"type":24,"tag":372,"props":1026,"children":1027},{"style":780},[1028],{"type":29,"value":994},{"type":24,"tag":372,"props":1030,"children":1031},{"style":791},[1032],{"type":29,"value":1024},{"type":24,"tag":372,"props":1034,"children":1035},{"style":780},[1036],{"type":29,"value":1037},"object_size ",{"type":24,"tag":372,"props":1039,"children":1040},{"style":791},[1041],{"type":29,"value":1042},"\u002F\u002F",{"type":24,"tag":372,"props":1044,"children":1045},{"style":780},[1046],{"type":29,"value":1047}," target_parts)))\n",{"type":24,"tag":25,"props":1049,"children":1050},{},[1051],{"type":29,"value":1052},"Ask for 5,000 parts rather than 10,000: it leaves room for a retry without re-planning.",{"type":24,"tag":25,"props":1054,"children":1055},{},[1056,1061],{"type":24,"tag":73,"props":1057,"children":1058},{},[1059],{"type":29,"value":1060},"2. Pass it to the client explicitly.",{"type":29,"value":1062}," The default is not a promise.",{"type":24,"tag":362,"props":1064,"children":1066},{"className":762,"code":1065,"language":764,"meta":7,"style":7},"from boto3.s3.transfer import TransferConfig\n\nsize = os.path.getsize(\"\u002Fbackups\u002Fdb-2026-11-06.tar\")\ncfg = TransferConfig(\n    multipart_threshold=8 * MIB,\n    multipart_chunksize=part_size(size),\n    max_concurrency=8,\n)\ns3.upload_file(\"\u002Fbackups\u002Fdb-2026-11-06.tar\", \"backups-prod\", \"db\u002F2026-11-06.tar\", Config=cfg)\n",[1067],{"type":24,"tag":32,"props":1068,"children":1069},{"__ignoreMap":7},[1070,1093,1100,1128,1145,1175,1192,1213,1221],{"type":24,"tag":372,"props":1071,"children":1072},{"class":374,"line":375},[1073,1078,1083,1088],{"type":24,"tag":372,"props":1074,"children":1075},{"style":791},[1076],{"type":29,"value":1077},"from",{"type":24,"tag":372,"props":1079,"children":1080},{"style":780},[1081],{"type":29,"value":1082}," boto3.s3.transfer ",{"type":24,"tag":372,"props":1084,"children":1085},{"style":791},[1086],{"type":29,"value":1087},"import",{"type":24,"tag":372,"props":1089,"children":1090},{"style":780},[1091],{"type":29,"value":1092}," TransferConfig\n",{"type":24,"tag":372,"props":1094,"children":1095},{"class":374,"line":830},[1096],{"type":24,"tag":372,"props":1097,"children":1098},{"emptyLinePlaceholder":906},[1099],{"type":29,"value":909},{"type":24,"tag":372,"props":1101,"children":1102},{"class":374,"line":902},[1103,1108,1113,1118,1123],{"type":24,"tag":372,"props":1104,"children":1105},{"style":780},[1106],{"type":29,"value":1107},"size ",{"type":24,"tag":372,"props":1109,"children":1110},{"style":791},[1111],{"type":29,"value":1112},"=",{"type":24,"tag":372,"props":1114,"children":1115},{"style":780},[1116],{"type":29,"value":1117}," os.path.getsize(",{"type":24,"tag":372,"props":1119,"children":1120},{"style":385},[1121],{"type":29,"value":1122},"\"\u002Fbackups\u002Fdb-2026-11-06.tar\"",{"type":24,"tag":372,"props":1124,"children":1125},{"style":780},[1126],{"type":29,"value":1127},")\n",{"type":24,"tag":372,"props":1129,"children":1130},{"class":374,"line":912},[1131,1136,1140],{"type":24,"tag":372,"props":1132,"children":1133},{"style":780},[1134],{"type":29,"value":1135},"cfg ",{"type":24,"tag":372,"props":1137,"children":1138},{"style":791},[1139],{"type":29,"value":1112},{"type":24,"tag":372,"props":1141,"children":1142},{"style":780},[1143],{"type":29,"value":1144}," TransferConfig(\n",{"type":24,"tag":372,"props":1146,"children":1147},{"class":374,"line":969},[1148,1153,1157,1162,1166,1170],{"type":24,"tag":372,"props":1149,"children":1150},{"style":379},[1151],{"type":29,"value":1152},"    multipart_threshold",{"type":24,"tag":372,"props":1154,"children":1155},{"style":791},[1156],{"type":29,"value":1112},{"type":24,"tag":372,"props":1158,"children":1159},{"style":774},[1160],{"type":29,"value":1161},"8",{"type":24,"tag":372,"props":1163,"children":1164},{"style":791},[1165],{"type":29,"value":877},{"type":24,"tag":372,"props":1167,"children":1168},{"style":774},[1169],{"type":29,"value":882},{"type":24,"tag":372,"props":1171,"children":1172},{"style":780},[1173],{"type":29,"value":1174},",\n",{"type":24,"tag":372,"props":1176,"children":1177},{"class":374,"line":978},[1178,1183,1187],{"type":24,"tag":372,"props":1179,"children":1180},{"style":379},[1181],{"type":29,"value":1182},"    multipart_chunksize",{"type":24,"tag":372,"props":1184,"children":1185},{"style":791},[1186],{"type":29,"value":1112},{"type":24,"tag":372,"props":1188,"children":1189},{"style":780},[1190],{"type":29,"value":1191},"part_size(size),\n",{"type":24,"tag":372,"props":1193,"children":1195},{"class":374,"line":1194},7,[1196,1201,1205,1209],{"type":24,"tag":372,"props":1197,"children":1198},{"style":379},[1199],{"type":29,"value":1200},"    max_concurrency",{"type":24,"tag":372,"props":1202,"children":1203},{"style":791},[1204],{"type":29,"value":1112},{"type":24,"tag":372,"props":1206,"children":1207},{"style":774},[1208],{"type":29,"value":1161},{"type":24,"tag":372,"props":1210,"children":1211},{"style":780},[1212],{"type":29,"value":1174},{"type":24,"tag":372,"props":1214,"children":1216},{"class":374,"line":1215},8,[1217],{"type":24,"tag":372,"props":1218,"children":1219},{"style":780},[1220],{"type":29,"value":1127},{"type":24,"tag":372,"props":1222,"children":1224},{"class":374,"line":1223},9,[1225,1230,1234,1238,1243,1247,1252,1256,1261,1265],{"type":24,"tag":372,"props":1226,"children":1227},{"style":780},[1228],{"type":29,"value":1229},"s3.upload_file(",{"type":24,"tag":372,"props":1231,"children":1232},{"style":385},[1233],{"type":29,"value":1122},{"type":24,"tag":372,"props":1235,"children":1236},{"style":780},[1237],{"type":29,"value":783},{"type":24,"tag":372,"props":1239,"children":1240},{"style":385},[1241],{"type":29,"value":1242},"\"backups-prod\"",{"type":24,"tag":372,"props":1244,"children":1245},{"style":780},[1246],{"type":29,"value":783},{"type":24,"tag":372,"props":1248,"children":1249},{"style":385},[1250],{"type":29,"value":1251},"\"db\u002F2026-11-06.tar\"",{"type":24,"tag":372,"props":1253,"children":1254},{"style":780},[1255],{"type":29,"value":783},{"type":24,"tag":372,"props":1257,"children":1258},{"style":379},[1259],{"type":29,"value":1260},"Config",{"type":24,"tag":372,"props":1262,"children":1263},{"style":791},[1264],{"type":29,"value":1112},{"type":24,"tag":372,"props":1266,"children":1267},{"style":780},[1268],{"type":29,"value":1269},"cfg)\n",{"type":24,"tag":25,"props":1271,"children":1272},{},[1273,1284],{"type":24,"tag":73,"props":1274,"children":1275},{},[1276,1278,1283],{"type":29,"value":1277},"3. Abort in a ",{"type":24,"tag":32,"props":1279,"children":1281},{"className":1280},[],[1282],{"type":29,"value":659},{"type":29,"value":745},{"type":29,"value":1285}," Parts survive a crashed process for as long as the upload ID does; nothing expires them.",{"type":24,"tag":362,"props":1287,"children":1289},{"className":762,"code":1288,"language":764,"meta":7,"style":7},"uid = s3.create_multipart_upload(Bucket=b, Key=k)[\"UploadId\"]\ntry:\n    parts = [...]            # upload_part calls, collected with their ETags\n    s3.complete_multipart_upload(Bucket=b, Key=k, UploadId=uid,\n                                 MultipartUpload={\"Parts\": parts})\nexcept Exception:\n    s3.abort_multipart_upload(Bucket=b, Key=k, UploadId=uid)\n    raise\n",[1290],{"type":24,"tag":32,"props":1291,"children":1292},{"__ignoreMap":7},[1293,1348,1360,1393,1440,1467,1484,1529],{"type":24,"tag":372,"props":1294,"children":1295},{"class":374,"line":375},[1296,1301,1305,1310,1315,1319,1324,1329,1333,1338,1343],{"type":24,"tag":372,"props":1297,"children":1298},{"style":780},[1299],{"type":29,"value":1300},"uid ",{"type":24,"tag":372,"props":1302,"children":1303},{"style":791},[1304],{"type":29,"value":1112},{"type":24,"tag":372,"props":1306,"children":1307},{"style":780},[1308],{"type":29,"value":1309}," s3.create_multipart_upload(",{"type":24,"tag":372,"props":1311,"children":1312},{"style":379},[1313],{"type":29,"value":1314},"Bucket",{"type":24,"tag":372,"props":1316,"children":1317},{"style":791},[1318],{"type":29,"value":1112},{"type":24,"tag":372,"props":1320,"children":1321},{"style":780},[1322],{"type":29,"value":1323},"b, ",{"type":24,"tag":372,"props":1325,"children":1326},{"style":379},[1327],{"type":29,"value":1328},"Key",{"type":24,"tag":372,"props":1330,"children":1331},{"style":791},[1332],{"type":29,"value":1112},{"type":24,"tag":372,"props":1334,"children":1335},{"style":780},[1336],{"type":29,"value":1337},"k)[",{"type":24,"tag":372,"props":1339,"children":1340},{"style":385},[1341],{"type":29,"value":1342},"\"UploadId\"",{"type":24,"tag":372,"props":1344,"children":1345},{"style":780},[1346],{"type":29,"value":1347},"]\n",{"type":24,"tag":372,"props":1349,"children":1350},{"class":374,"line":830},[1351,1356],{"type":24,"tag":372,"props":1352,"children":1353},{"style":791},[1354],{"type":29,"value":1355},"try",{"type":24,"tag":372,"props":1357,"children":1358},{"style":780},[1359],{"type":29,"value":966},{"type":24,"tag":372,"props":1361,"children":1362},{"class":374,"line":902},[1363,1368,1372,1377,1382,1387],{"type":24,"tag":372,"props":1364,"children":1365},{"style":780},[1366],{"type":29,"value":1367},"    parts ",{"type":24,"tag":372,"props":1369,"children":1370},{"style":791},[1371],{"type":29,"value":1112},{"type":24,"tag":372,"props":1373,"children":1374},{"style":780},[1375],{"type":29,"value":1376}," [",{"type":24,"tag":372,"props":1378,"children":1379},{"style":774},[1380],{"type":29,"value":1381},"...",{"type":24,"tag":372,"props":1383,"children":1384},{"style":780},[1385],{"type":29,"value":1386},"]            ",{"type":24,"tag":372,"props":1388,"children":1390},{"style":1389},"--shiki-default:#66707B;--shiki-dark:#BDC4CC",[1391],{"type":29,"value":1392},"# upload_part calls, collected with their ETags\n",{"type":24,"tag":372,"props":1394,"children":1395},{"class":374,"line":912},[1396,1401,1405,1409,1413,1417,1421,1426,1431,1435],{"type":24,"tag":372,"props":1397,"children":1398},{"style":780},[1399],{"type":29,"value":1400},"    s3.complete_multipart_upload(",{"type":24,"tag":372,"props":1402,"children":1403},{"style":379},[1404],{"type":29,"value":1314},{"type":24,"tag":372,"props":1406,"children":1407},{"style":791},[1408],{"type":29,"value":1112},{"type":24,"tag":372,"props":1410,"children":1411},{"style":780},[1412],{"type":29,"value":1323},{"type":24,"tag":372,"props":1414,"children":1415},{"style":379},[1416],{"type":29,"value":1328},{"type":24,"tag":372,"props":1418,"children":1419},{"style":791},[1420],{"type":29,"value":1112},{"type":24,"tag":372,"props":1422,"children":1423},{"style":780},[1424],{"type":29,"value":1425},"k, ",{"type":24,"tag":372,"props":1427,"children":1428},{"style":379},[1429],{"type":29,"value":1430},"UploadId",{"type":24,"tag":372,"props":1432,"children":1433},{"style":791},[1434],{"type":29,"value":1112},{"type":24,"tag":372,"props":1436,"children":1437},{"style":780},[1438],{"type":29,"value":1439},"uid,\n",{"type":24,"tag":372,"props":1441,"children":1442},{"class":374,"line":969},[1443,1448,1452,1457,1462],{"type":24,"tag":372,"props":1444,"children":1445},{"style":379},[1446],{"type":29,"value":1447},"                                 MultipartUpload",{"type":24,"tag":372,"props":1449,"children":1450},{"style":791},[1451],{"type":29,"value":1112},{"type":24,"tag":372,"props":1453,"children":1454},{"style":780},[1455],{"type":29,"value":1456},"{",{"type":24,"tag":372,"props":1458,"children":1459},{"style":385},[1460],{"type":29,"value":1461},"\"Parts\"",{"type":24,"tag":372,"props":1463,"children":1464},{"style":780},[1465],{"type":29,"value":1466},": parts})\n",{"type":24,"tag":372,"props":1468,"children":1469},{"class":374,"line":978},[1470,1475,1480],{"type":24,"tag":372,"props":1471,"children":1472},{"style":791},[1473],{"type":29,"value":1474},"except",{"type":24,"tag":372,"props":1476,"children":1477},{"style":774},[1478],{"type":29,"value":1479}," Exception",{"type":24,"tag":372,"props":1481,"children":1482},{"style":780},[1483],{"type":29,"value":966},{"type":24,"tag":372,"props":1485,"children":1486},{"class":374,"line":1194},[1487,1492,1496,1500,1504,1508,1512,1516,1520,1524],{"type":24,"tag":372,"props":1488,"children":1489},{"style":780},[1490],{"type":29,"value":1491},"    s3.abort_multipart_upload(",{"type":24,"tag":372,"props":1493,"children":1494},{"style":379},[1495],{"type":29,"value":1314},{"type":24,"tag":372,"props":1497,"children":1498},{"style":791},[1499],{"type":29,"value":1112},{"type":24,"tag":372,"props":1501,"children":1502},{"style":780},[1503],{"type":29,"value":1323},{"type":24,"tag":372,"props":1505,"children":1506},{"style":379},[1507],{"type":29,"value":1328},{"type":24,"tag":372,"props":1509,"children":1510},{"style":791},[1511],{"type":29,"value":1112},{"type":24,"tag":372,"props":1513,"children":1514},{"style":780},[1515],{"type":29,"value":1425},{"type":24,"tag":372,"props":1517,"children":1518},{"style":379},[1519],{"type":29,"value":1430},{"type":24,"tag":372,"props":1521,"children":1522},{"style":791},[1523],{"type":29,"value":1112},{"type":24,"tag":372,"props":1525,"children":1526},{"style":780},[1527],{"type":29,"value":1528},"uid)\n",{"type":24,"tag":372,"props":1530,"children":1531},{"class":374,"line":1215},[1532],{"type":24,"tag":372,"props":1533,"children":1534},{"style":791},[1535],{"type":29,"value":1536},"    raise\n",{"type":24,"tag":25,"props":1538,"children":1539},{},[1540,1545],{"type":24,"tag":73,"props":1541,"children":1542},{},[1543],{"type":29,"value":1544},"4. Prove the limit on the endpoint you deploy against.",{"type":29,"value":1546}," Run the probe against a test bucket: a 1 MiB part must be rejected, a 5 MiB part accepted. Two calls answer what no documentation can: whether this provider enforces the floor at all.",{"type":24,"tag":362,"props":1548,"children":1550},{"className":364,"code":1549,"language":366,"meta":7,"style":7},"python3 scripts\u002Fmultipart-probe.py probe https:\u002F\u002Fs3.eu-west-3.amazonaws.com\n",[1551],{"type":24,"tag":32,"props":1552,"children":1553},{"__ignoreMap":7},[1554],{"type":24,"tag":372,"props":1555,"children":1556},{"class":374,"line":375},[1557,1561,1565,1569],{"type":24,"tag":372,"props":1558,"children":1559},{"style":379},[1560],{"type":29,"value":382},{"type":24,"tag":372,"props":1562,"children":1563},{"style":385},[1564],{"type":29,"value":388},{"type":24,"tag":372,"props":1566,"children":1567},{"style":385},[1568],{"type":29,"value":567},{"type":24,"tag":372,"props":1570,"children":1571},{"style":385},[1572],{"type":29,"value":1573}," https:\u002F\u002Fs3.eu-west-3.amazonaws.com\n",{"type":24,"tag":25,"props":1575,"children":1576},{},[1577,1582],{"type":24,"tag":73,"props":1578,"children":1579},{},[1580],{"type":29,"value":1581},"5. Verify what landed with the composite rule.",{"type":29,"value":1583}," Compare the ETag to the value computed from the part MD5s, never to the MD5 of the file.",{"type":24,"tag":362,"props":1585,"children":1587},{"className":762,"code":1586,"language":764,"meta":7,"style":7},"import hashlib\nmd5s = b\"\".join(hashlib.md5(chunk).digest() for chunk in parts)\nexpected = f'\"{hashlib.md5(md5s).hexdigest()}-{len(parts)}\"'\nassert s3.head_object(Bucket=b, Key=k)[\"ETag\"] == expected\n",[1588],{"type":24,"tag":32,"props":1589,"children":1590},{"__ignoreMap":7},[1591,1603,1650,1713],{"type":24,"tag":372,"props":1592,"children":1593},{"class":374,"line":375},[1594,1598],{"type":24,"tag":372,"props":1595,"children":1596},{"style":791},[1597],{"type":29,"value":1087},{"type":24,"tag":372,"props":1599,"children":1600},{"style":780},[1601],{"type":29,"value":1602}," hashlib\n",{"type":24,"tag":372,"props":1604,"children":1605},{"class":374,"line":830},[1606,1611,1615,1620,1625,1630,1635,1640,1645],{"type":24,"tag":372,"props":1607,"children":1608},{"style":780},[1609],{"type":29,"value":1610},"md5s ",{"type":24,"tag":372,"props":1612,"children":1613},{"style":791},[1614],{"type":29,"value":1112},{"type":24,"tag":372,"props":1616,"children":1617},{"style":791},[1618],{"type":29,"value":1619}," b",{"type":24,"tag":372,"props":1621,"children":1622},{"style":385},[1623],{"type":29,"value":1624},"\"\"",{"type":24,"tag":372,"props":1626,"children":1627},{"style":780},[1628],{"type":29,"value":1629},".join(hashlib.md5(chunk).digest() ",{"type":24,"tag":372,"props":1631,"children":1632},{"style":791},[1633],{"type":29,"value":1634},"for",{"type":24,"tag":372,"props":1636,"children":1637},{"style":780},[1638],{"type":29,"value":1639}," chunk ",{"type":24,"tag":372,"props":1641,"children":1642},{"style":791},[1643],{"type":29,"value":1644},"in",{"type":24,"tag":372,"props":1646,"children":1647},{"style":780},[1648],{"type":29,"value":1649}," parts)\n",{"type":24,"tag":372,"props":1651,"children":1652},{"class":374,"line":902},[1653,1658,1662,1667,1672,1676,1681,1686,1690,1694,1699,1704,1708],{"type":24,"tag":372,"props":1654,"children":1655},{"style":780},[1656],{"type":29,"value":1657},"expected ",{"type":24,"tag":372,"props":1659,"children":1660},{"style":791},[1661],{"type":29,"value":1112},{"type":24,"tag":372,"props":1663,"children":1664},{"style":791},[1665],{"type":29,"value":1666}," f",{"type":24,"tag":372,"props":1668,"children":1669},{"style":385},[1670],{"type":29,"value":1671},"'\"",{"type":24,"tag":372,"props":1673,"children":1674},{"style":791},[1675],{"type":29,"value":1456},{"type":24,"tag":372,"props":1677,"children":1678},{"style":780},[1679],{"type":29,"value":1680},"hashlib.md5(md5s).hexdigest()",{"type":24,"tag":372,"props":1682,"children":1683},{"style":791},[1684],{"type":29,"value":1685},"}",{"type":24,"tag":372,"props":1687,"children":1688},{"style":385},[1689],{"type":29,"value":1024},{"type":24,"tag":372,"props":1691,"children":1692},{"style":791},[1693],{"type":29,"value":1456},{"type":24,"tag":372,"props":1695,"children":1696},{"style":774},[1697],{"type":29,"value":1698},"len",{"type":24,"tag":372,"props":1700,"children":1701},{"style":780},[1702],{"type":29,"value":1703},"(parts)",{"type":24,"tag":372,"props":1705,"children":1706},{"style":791},[1707],{"type":29,"value":1685},{"type":24,"tag":372,"props":1709,"children":1710},{"style":385},[1711],{"type":29,"value":1712},"\"'\n",{"type":24,"tag":372,"props":1714,"children":1715},{"class":374,"line":912},[1716,1721,1726,1730,1734,1738,1742,1746,1750,1755,1760,1765],{"type":24,"tag":372,"props":1717,"children":1718},{"style":791},[1719],{"type":29,"value":1720},"assert",{"type":24,"tag":372,"props":1722,"children":1723},{"style":780},[1724],{"type":29,"value":1725}," s3.head_object(",{"type":24,"tag":372,"props":1727,"children":1728},{"style":379},[1729],{"type":29,"value":1314},{"type":24,"tag":372,"props":1731,"children":1732},{"style":791},[1733],{"type":29,"value":1112},{"type":24,"tag":372,"props":1735,"children":1736},{"style":780},[1737],{"type":29,"value":1323},{"type":24,"tag":372,"props":1739,"children":1740},{"style":379},[1741],{"type":29,"value":1328},{"type":24,"tag":372,"props":1743,"children":1744},{"style":791},[1745],{"type":29,"value":1112},{"type":24,"tag":372,"props":1747,"children":1748},{"style":780},[1749],{"type":29,"value":1337},{"type":24,"tag":372,"props":1751,"children":1752},{"style":385},[1753],{"type":29,"value":1754},"\"ETag\"",{"type":24,"tag":372,"props":1756,"children":1757},{"style":780},[1758],{"type":29,"value":1759},"] ",{"type":24,"tag":372,"props":1761,"children":1762},{"style":791},[1763],{"type":29,"value":1764},"==",{"type":24,"tag":372,"props":1766,"children":1767},{"style":780},[1768],{"type":29,"value":1769}," expected\n",{"type":24,"tag":41,"props":1771,"children":1773},{"id":1772},"how-s3-explorer-fits",[1774],{"type":29,"value":1775},"How S3 Explorer fits",{"type":24,"tag":25,"props":1777,"children":1778},{},[1779,1781,1786,1788,1793,1795,1800,1802,1808,1809,1815,1817,1825],{"type":29,"value":1780},"These limits are protocol-level, so the useful check runs on the endpoint you actually use, often not the one in the ticket. S3 Explorer's ",{"type":24,"tag":73,"props":1782,"children":1783},{},[1784],{"type":29,"value":1785},"provider picker",{"type":29,"value":1787}," offers 25 presets across Cloud, Specialist, Self-hosted and Enterprise plus a Custom tile for any endpoint, region and addressing style, so one bucket can be opened on AWS and on a self-hosted server from the same window. The ",{"type":24,"tag":73,"props":1789,"children":1790},{},[1791],{"type":29,"value":1792},"test connection",{"type":29,"value":1794}," step reports how many buckets the credentials can see before a profile is saved, and the ",{"type":24,"tag":73,"props":1796,"children":1797},{},[1798],{"type":29,"value":1799},"per-request metrics",{"type":29,"value":1801}," view counts the ",{"type":24,"tag":32,"props":1803,"children":1805},{"className":1804},[],[1806],{"type":29,"value":1807},"LIST",{"type":29,"value":783},{"type":24,"tag":32,"props":1810,"children":1812},{"className":1811},[],[1813],{"type":29,"value":1814},"GET",{"type":29,"value":1816}," and other calls a session generated. ",{"type":24,"tag":730,"props":1818,"children":1822},{"href":1819,"rel":1820},"https:\u002F\u002Fs3explorer.accessdevops.com",[1821],"nofollow",[1823],{"type":29,"value":1824},"S3 Explorer",{"type":29,"value":1826}," runs on Windows, macOS and Linux.",{"type":24,"tag":41,"props":1828,"children":1830},{"id":1829},"checklist",[1831],{"type":29,"value":1832},"Checklist",{"type":24,"tag":48,"props":1834,"children":1837},{"className":1835},[1836],"contains-task-list",[1838,1850,1859,1868,1893,1916,1925],{"type":24,"tag":52,"props":1839,"children":1842},{"className":1840},[1841],"task-list-item",[1843,1848],{"type":24,"tag":1844,"props":1845,"children":1847},"input",{"disabled":906,"type":1846},"checkbox",[],{"type":29,"value":1849}," Every upload path computes its part size from the object size, floored at 5 MiB and capped at 5 GiB.",{"type":24,"tag":52,"props":1851,"children":1853},{"className":1852},[1841],[1854,1857],{"type":24,"tag":1844,"props":1855,"children":1856},{"disabled":906,"type":1846},[],{"type":29,"value":1858}," No path relies on the SDK default part size for objects above 78 GiB.",{"type":24,"tag":52,"props":1860,"children":1862},{"className":1861},[1841],[1863,1866],{"type":24,"tag":1844,"props":1864,"children":1865},{"disabled":906,"type":1846},[],{"type":29,"value":1867}," The final part keeps its real size, not padded to the minimum.",{"type":24,"tag":52,"props":1869,"children":1871},{"className":1870},[1841],[1872,1875,1877,1883,1885,1891],{"type":24,"tag":1844,"props":1873,"children":1874},{"disabled":906,"type":1846},[],{"type":29,"value":1876}," ",{"type":24,"tag":32,"props":1878,"children":1880},{"className":1879},[],[1881],{"type":29,"value":1882},"EntityTooSmall",{"type":29,"value":1884}," and ",{"type":24,"tag":32,"props":1886,"children":1888},{"className":1887},[],[1889],{"type":29,"value":1890},"InvalidPart",{"type":29,"value":1892}," are reported as different causes, not one \"400\".",{"type":24,"tag":52,"props":1894,"children":1896},{"className":1895},[1841],[1897,1900,1902,1908,1910,1915],{"type":24,"tag":1844,"props":1898,"children":1899},{"disabled":906,"type":1846},[],{"type":29,"value":1901}," Every multipart block has an ",{"type":24,"tag":32,"props":1903,"children":1905},{"className":1904},[],[1906],{"type":29,"value":1907},"abort_multipart_upload",{"type":29,"value":1909}," in a ",{"type":24,"tag":32,"props":1911,"children":1913},{"className":1912},[],[1914],{"type":29,"value":659},{"type":29,"value":745},{"type":24,"tag":52,"props":1917,"children":1919},{"className":1918},[1841],[1920,1923],{"type":24,"tag":1844,"props":1921,"children":1922},{"disabled":906,"type":1846},[],{"type":29,"value":1924}," A lifecycle rule aborts incomplete multipart uploads after N days, and N is written down.",{"type":24,"tag":52,"props":1926,"children":1928},{"className":1927},[1841],[1929,1932,1934,1940],{"type":24,"tag":1844,"props":1930,"children":1931},{"disabled":906,"type":1846},[],{"type":29,"value":1933}," Integrity checks compare the composite ETag with its ",{"type":24,"tag":32,"props":1935,"children":1937},{"className":1936},[],[1938],{"type":29,"value":1939},"-N",{"type":29,"value":1941}," suffix, or the checksum headers.",{"type":24,"tag":41,"props":1943,"children":1945},{"id":1944},"sources",[1946],{"type":29,"value":1947},"Sources",{"type":24,"tag":48,"props":1949,"children":1950},{},[1951,1963,1982,2002,2021,2041,2092],{"type":24,"tag":52,"props":1952,"children":1953},{},[1954,1961],{"type":24,"tag":730,"props":1955,"children":1958},{"href":1956,"rel":1957},"https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonS3\u002Flatest\u002Fuserguide\u002Fqfacts.html",[1821],[1959],{"type":29,"value":1960},"Amazon S3 — multipart upload limits",{"type":29,"value":1962},": 10,000 parts, part numbers 1–10,000, part size 5 MiB–5 GiB, \"no minimum size limit on the last part\", maximum object size 48.8 TiB.",{"type":24,"tag":52,"props":1964,"children":1965},{},[1966,1973,1975,1980],{"type":24,"tag":730,"props":1967,"children":1970},{"href":1968,"rel":1969},"https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonS3\u002Flatest\u002Fuserguide\u002Fupload-objects.html",[1821],[1971],{"type":29,"value":1972},"Amazon S3 — uploading objects",{"type":29,"value":1974},": single ",{"type":24,"tag":32,"props":1976,"children":1978},{"className":1977},[],[1979],{"type":29,"value":37},{"type":29,"value":1981}," \"up to 5 GB\", console uploads to 160 GB, multipart \"up to 50 TB\".",{"type":24,"tag":52,"props":1983,"children":1984},{},[1985,1992,1994,2000],{"type":24,"tag":730,"props":1986,"children":1989},{"href":1987,"rel":1988},"https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonS3\u002Flatest\u002FAPI\u002FAPI_CompleteMultipartUpload.html",[1821],[1990],{"type":29,"value":1991},"Amazon S3 — CompleteMultipartUpload",{"type":29,"value":1993},": parts are concatenated in ascending part-number order; a ",{"type":24,"tag":32,"props":1995,"children":1997},{"className":1996},[],[1998],{"type":29,"value":1999},"200 OK",{"type":29,"value":2001}," can still carry an error inside the body.",{"type":24,"tag":52,"props":2003,"children":2004},{},[2005,2012,2014,2019],{"type":24,"tag":730,"props":2006,"children":2009},{"href":2007,"rel":2008},"https:\u002F\u002Fdocs.aws.amazon.com\u002FAmazonS3\u002Flatest\u002FAPI\u002FAPI_Error.html",[1821],[2010],{"type":29,"value":2011},"Amazon S3 — error code catalogue",{"type":29,"value":2013},": ",{"type":24,"tag":32,"props":2015,"children":2017},{"className":2016},[],[2018],{"type":29,"value":1882},{"type":29,"value":2020}," — \"Your proposed upload is smaller than the minimum allowed object size\".",{"type":24,"tag":52,"props":2022,"children":2023},{},[2024,2031,2032,2039],{"type":24,"tag":730,"props":2025,"children":2028},{"href":2026,"rel":2027},"https:\u002F\u002Fdevelopers.cloudflare.com\u002Fr2\u002Fplatform\u002Flimits\u002F",[1821],[2029],{"type":29,"value":2030},"Cloudflare R2 — limits",{"type":29,"value":1884},{"type":24,"tag":730,"props":2033,"children":2036},{"href":2034,"rel":2035},"https:\u002F\u002Fdevelopers.cloudflare.com\u002Fr2\u002Fobjects\u002Fupload-objects\u002F",[1821],[2037],{"type":29,"value":2038},"upload objects",{"type":29,"value":2040},": multipart objects to 5 TiB, 10,000 parts, part size 5 MiB–5 GiB, single upload 5 GiB.",{"type":24,"tag":52,"props":2042,"children":2043},{},[2044,2062,2063,2069,2070,2076,2077,2083,2085,2090],{"type":24,"tag":730,"props":2045,"children":2048},{"href":2046,"rel":2047},"https:\u002F\u002Fgithub.com\u002Fboto\u002Fs3transfer\u002Fblob\u002Fdevelop\u002Fs3transfer\u002Futils.py",[1821],[2049,2054,2056],{"type":24,"tag":32,"props":2050,"children":2052},{"className":2051},[],[2053],{"type":29,"value":156},{"type":29,"value":2055}," 0.19.3 — ",{"type":24,"tag":32,"props":2057,"children":2059},{"className":2058},[],[2060],{"type":29,"value":2061},"utils.py",{"type":29,"value":2013},{"type":24,"tag":32,"props":2064,"children":2066},{"className":2065},[],[2067],{"type":29,"value":2068},"MAX_PARTS = 10000",{"type":29,"value":783},{"type":24,"tag":32,"props":2071,"children":2073},{"className":2072},[],[2074],{"type":29,"value":2075},"MIN_UPLOAD_CHUNKSIZE = 5 * (1024**2)",{"type":29,"value":783},{"type":24,"tag":32,"props":2078,"children":2080},{"className":2079},[],[2081],{"type":29,"value":2082},"MAX_SINGLE_UPLOAD_SIZE = 5 * (1024**3)",{"type":29,"value":2084},", and the ",{"type":24,"tag":32,"props":2086,"children":2088},{"className":2087},[],[2089],{"type":29,"value":172},{"type":29,"value":2091}," doubling loop.",{"type":24,"tag":52,"props":2093,"children":2094},{},[2095,2097,2102,2104,2109,2111,2117],{"type":29,"value":2096},"Measured 6 November 2026 against SeaweedFS 4.50 (S3 gateway) and moto 5.2.5 with ",{"type":24,"tag":32,"props":2098,"children":2100},{"className":2099},[],[2101],{"type":29,"value":439},{"type":29,"value":2103}," 1.43.125; both returned ",{"type":24,"tag":32,"props":2105,"children":2107},{"className":2106},[],[2108],{"type":29,"value":1882},{"type":29,"value":2110}," for a 1 MiB non-final part. The probe is ",{"type":24,"tag":32,"props":2112,"children":2114},{"className":2113},[],[2115],{"type":29,"value":2116},"scripts\u002Fmultipart-probe.py",{"type":29,"value":2118}," in this repository.",{"type":24,"tag":2120,"props":2121,"children":2122},"style",{},[2123],{"type":29,"value":2124},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":7,"searchDepth":830,"depth":830,"links":2126},[2127,2128,2129,2130,2131,2132,2133],{"id":43,"depth":830,"text":46},{"id":111,"depth":830,"text":114},{"id":409,"depth":830,"text":412},{"id":748,"depth":830,"text":751},{"id":1772,"depth":830,"text":1775},{"id":1829,"depth":830,"text":1832},{"id":1944,"depth":830,"text":1947},"markdown","content:blog:multipart-5mb-floor-10000-parts.md","content","blog\u002Fmultipart-5mb-floor-10000-parts.md","blog\u002Fmultipart-5mb-floor-10000-parts","md",[2141,2149,2156,2162,2164,2168,2174,2179,2187,2194],{"_path":2142,"title":2143,"description":2144,"tags":2145},"\u002Fblog\u002Fabac-object-tags-vs-prefix-policies","Object tags or prefixes: which access model survives growth","Prefixes decide what a principal can list, tags decide what it can read. Two AWS limits break tag-only designs, and prefix policies run out at about 60 tenants.",[13,2146,2147,2148],"iam","governance","access-control",{"_path":2150,"title":2151,"description":2152,"tags":2153},"\u002Fblog\u002Fassumerole-chaining-one-hour-ceiling","Chained AssumeRole sessions stop at one hour, silently","A role that assumes another role gets 3600 seconds, whatever its MaxSessionDuration allows. The exact error, and the batch job that dies at minute 61.",[13,2146,2154,2155],"sts","aws",{"_path":2157,"title":2158,"description":2159,"tags":2160},"\u002Fblog\u002Fbucket-policy-allow-still-denied","When two policies allow and the request still fails","S3 returns AccessDenied even when the bucket and IAM policies both allow. Here is the five-gate evaluation order and where the hidden deny sits.",[13,2146,2161,2148],"security",{"_path":4,"title":8,"description":9,"tags":2163},[13,14,15],{"_path":740,"title":2165,"description":2166,"tags":2167},"Presigned POST: cap size and type before the upload starts","A presigned PUT signs a URL, not a byte count. A POST policy adds a size range and a content-type condition that S3 enforces while the body arrives.",[13,2161,14],{"_path":2169,"title":2170,"description":2171,"tags":2172},"\u002Fblog\u002Fpresigned-url-leakage","Presigned URLs leak into logs, referrers and chat","A presigned URL is a working credential stored as text. Where the copy ends up, what it still grants after redaction, and the 15-minute fix.",[13,2161,2173],"presigned-urls",{"_path":2175,"title":2176,"description":2177,"tags":2178},"\u002Fblog\u002Fpresigned-url-seven-day-ceiling","Presigned URLs expire at seven days. Then what?","Seven days is not a setting: it is how long the SigV4 signing key lives. What happens at that wall, and the three patterns that outlive it.",[13,2161,2173],{"_path":2180,"title":2181,"description":2182,"tags":2183},"\u002Fblog\u002Fs3-explorer-walkthrough","S3 Explorer walkthrough: zero to browsing in 5 minutes","A step-by-step walkthrough of S3 Explorer against a live S3 endpoint: pick a provider, test, browse buckets, inspect metadata, read request metrics.",[13,2184,2185,2186],"tutorial","tooling","seaweedfs",{"_path":732,"title":2188,"description":2189,"tags":2190},"The real cost of S3: 8 providers compared, egress included","Storage is 20% of your bill. We modelled 8 S3 providers across 4 workloads — backup, app, media, DR — with storage, egress, requests and replication.",[13,2191,2192,2193],"pricing","cloud","cost-optimisation",{"_path":2195,"title":2196,"description":2197,"tags":2198},"\u002Fblog\u002Fwelcome","Why we write about S3, and how this blog is made","Object storage is everywhere and most advice about it is stale. What this blog covers, how each article is sourced, and what we measure to know it helped.",[13,2192],1793964733897]