![\"An](\"media/huge-on-the-tray.jpeg\")
Here's a fun one!\nYou may already know that tarball is a pure archive format,\nand that any compression is applied to the whole archive as a unit.\nThat is to say that compression is not actually applied at the file level,\nbut to the entire archive.
\nThis is a trade-off the designers made to limit complexity,\nand as a side-effect, is the reason why you can't randomly access parts of a compressed tarball.
\nWhat you may not know is that the tar utility has built-in support for a few formats!\nGZIP is probably the most commonly used for historical reasons,\nbut zstd and lz4 are built-in options on my Mac.\nThis is probably system-dependent, so check your local manpages.
Here's an example of compressing and decompressing with zstd:
tar --zstd -cf directory.tar.zst directory/\ntar --zstd -xf directory.tar.zst\nYou can also use this with any (de)compression program that operates on stdin and stdout!
\ntar --use-compress-program zstd -cf directory.tar.zst directory/\nPretty cool, huh?\nIt's no different that using pipes at the end of the day,\nbut it does simplify the invocation a bit in my opinion.
\nAfter I initially published this article,\n@cartocalypse@norden.social noted that some versions of tar include the\n-a/--auto-compress option which will automatically determine format and compression based on the suffix!\nCheck your manpages for details; it appears to work on FreeBSD, macOS (which inherits the FreeBSD implementation), and GNU tar.
I've been using nushell as my daily driver for about six months now,\nand wanted to show a few simple examples of why I'm enjoying it so much.\nI think it's a breath of fresh air compared to most shells.
\nIn case you've never heard of it before, nushell is a, well, new shell ;)\nbash has been the dominant shell for as long as I can remember,\nthough zsh have their fair share of devotees.\nfish is the only recent example I can think of as a "challenger" shell.\nfish gained enough traction that it's supported by tooling such as Python virtualenv\n(which only has integrations out of the box for a handful of shells).\nI think fish is popular because it had some slightly saner defaults out of the box,\nwas easier to "customize" with flashy prompts (which can make your shell SUPER slow to init),\nand had a saner scripting language than bash.\nBut it still retained a lot of the historical baggage from POSIX shells.
Nushell challenges two common assumptions about shells\nand asks "what if things were different?"
\nls and du) are replaced by builtins.By dropping the goal of POSIX compliance,\nnushell frees itself from decades of baggage.\nThis means you get a scripting language that feels a lot more like Rust.\nYou'll actually get errors by default when you try to do something stupid,\nunlike most shells which will happily proceed,\nusually doing something even more stupid.\nMaybe treating undefined variables as empty string make sense in the 1970s,\nbut that's almost never helpful.
\nnushell also takes a relatively unique approach to utilities.\nWhen you type something like ls or ps in nushell,\nthis is handled by a shell builtin!\nIt's just Rust code baked into the shell rather than calling out to GNU coreutils\nor whatever your base system includes.\nThis means that whether you type ps on FreeBSD, Debian, or macOS,\nyou'll get the same behavior!
I can already hear some readers thinking "doesn't this just massively bloat the shell?"\nNo, not really.\nThe code for these is for less than that of the typical GNU utility,\nbecause nushell actually (IMO) embraces UNIX philosophy even better than the original utilities.\nThey are all extremely minimal and work with other builtins.\nFor example, there are no sorting flags for ls,\nand no format/unit flags for du.
The reason that nushell can take this approach is because they challenge the notion that\n"text is the universal API."\nYou can't meaningfully manipulate text without lots of lossy heuristics.\nBut you can do this for structured data!\nI admit I'm a bit of a chaos monkey, so I love to see a project taking a rare new approach\nin space where nothing has fundamentally changed since the 1970s.
\nOkay, enough about philosophy... here are a few examples of some shell pipelines I found delightful.
\nFirst up: I do a lot of work with Elasticsearch during $DAYJOB.\nOne workflow I have to do fairly often is spin up a cluster and restore from a snapshot.\nThe Elasticsearch API is great... for programs.\nBut I have a hard time grokking hundreds of lines of JSON.\nHere's an example of a pipeline I built which culls the JSON response down to just the section I care about.
http get "http://localhost:9200/myindex/_recovery"\n | get restored-indexname\n | get shards\n | get index\n | get size\nIn case it's not obvious, the http command makes HTTP requests.\nThis is another nushell builtin that is an excellent alternative to curl.\nIt's not as feature-rich (curl has a few decades of head start),\nbut it brings something new to the table: it understands from the response that the content is JSON,\nand converts it into structured data!\nEverything in this pipeline is nushell builtins.\nAnd it'll work on any OS that nushell supports.\nEven Windows!\nThat's wild!
Pro tip: you can press option+enter to add a new line when typing in the shell.
\nHere's an example I hinted at earlier.\nIf you type help du (to get built-in docs),\nyou won't find any flags for changing the units.\nBut you can do it using formatters like so:
du path/to/bigfile.bin | format filesize B apparent\nThe du command always shows human-readable units by default. Which I very much appreciate!\nAnd did you notice apparent at the end there?\nWell, the version of du you'd find with a typical Linux distro doesn't exactly lie to you,\nbut it withholds some very important information.\nThe physical size occupied on disk is not necessarily the same as how large the file\n(in an abstract platonic sense) actually is.
There are a bunch of reasons for this, but the most impactful one is compressed filesystems.\nIf I ask Linux du how large a file is in an OpenZFS dataset,\nit will report the physical size by default, which may be a few hundred megabytes\nwhen the file is really multiple gigabytes.\nNot necessarily helpful.
Anyways, the nushell builtin always gives you columns for both physical and apparent.\nSo you can't ignore the fact that these sizes are often different.\nI like that!
\nIf you want to give nushell a try,\nthey have some great documentation.\nRead the basics, but also check out their specific pages on, e.g.\ncoming from bash.
\nFinally, here are two more that tripped me up at first.
\n$nu.pid instead of $$.$env. On the plus side, you can now explicitly differentiate from environment variables.If you're a developer or sysadmin, there's a pretty good chance you've had to transfer files back and forth.\nBack in the old days, you may have used the ftp utility or something similar\n(I think my first one was probably CuteFTP).\nThen you probably thought better of doing things in plaintext,\nand switched to SFTP/SCP, which operate over an SSH connection.
My quick post today is not exactly a bombshell of new information,\nbut these tools are not the fastest way to transfer.\nThere's often not a huge difference if you're transferring between machines in the same datacenter,\nbut you can do many times better then transferring from home or the office.
\nOf course I'm talking about rsync, which has a lot of features like compression, partial transfer resume,\nchecksumming so just the deltas are sent in a large file, and more.\nI don't know why, but it's rarely in my consciousness, and always strikes me as a bit quirky.\nYou need quite a few flags for what I would consider to be reasonable defaults.\nBut if you remember those (or use shel history, like me),\nit can save you a ton of time.
In fact, this morning, using rsync saved me more time than it took to write this blog post.\nI was transferring a ~40GB file from a server halfway around the world,\nbut I only had to transfer bytes equivalent to 20% of the total.
\nHere's a look at the rsync command I often use for pulling files from a remote server\n(I usually do this to download a large build artifact without going through a cloud storage intermediary,\nwhich is both slower and more eexpensive):
rsync -Pzhv example.com:/remote/path/to/big/file ~/file\nIt's not really that bad compared to an scp invocation, but those flags make all the difference.\nHere's what they do:
-P - Keeps partially transferred files (in case of interruption) and shows progress during the transfer.-z - Compresses data on the source server before sending to the destination. This probably isn't a great idea for an intra-datacenter transfer (it just wastes CPU), but it's perfect for long distance transfers over "slower" links (where I'd say slower is something less than like 100Mbps between you and the server... the last part is important because you may have a gigabit link, but the peering arrangements or other issues conspire to limit effective transfer rates to something much lower).-h - Makes the output more "human-readable." This shows previfexs like K, M, or G. You can add it twice if you'd like 1024-based values instead of 1000-based. To be honest, I don't know why this isn't the default.-v - Verbose mode. By default, rsync is silent. This is another behavior that I find strange in the present, but probably made more sense in the era of teletype terminals and very slow links. It's not really that verbose; it just tells you what files are being transferred and a brief summary at the end. You actually have to give two v's (-vv) for rsync to tell you which files it's skipping!Hope this helps speed up your next remote file transfer.\nIf there's any options you like which I may have missed, hit me up on Mastodon with a suggestion!
\nBonus pro tip: I had (until I recently switched to nushell) over a decade of accumulated shell history,\nand sometimes it's hard to keep the options straight.\nNaturally my history has a few different variations on a command like rsync.\nRather than searching through manpages with the equivalent of grep,\nI usually go to explainshell.com.\nIt seems to be a frontend to the manapages that understands the various sections,\nproviding a much quicker explanation of what your switches do!
It occurred to me that things which work well are often under-represented online.\nI absolutely love my listening headphones, so I thought I'd write a quick post about them.
\nTo say I love music is an understatement.\nMy iTunes... er... Apple Music library is has 61.5 days of it.\nAnd I am reasonably picky.\nAnd unlike my Steam library, I have listened to all of it!\nMany times over.
\nI also tend to listen a lot in situations where headphones are the best output device.\nEither nobody else is awake (I'm listening to Arthur Rbenstein's performance of Chopin's Nocturnes right now, if that gives you a clue)\nor others are very much around who may not appreciate classic trance blasting at 100dB.\nHeadphones are a really fantastic way to listen to music,\nand if you care about quality, they are some of the best bang for your buck.\nYou can get some AMAZING quality cans for a fraction of what you'd pay for a comparable speaker setup.
\nBut headphones are also hard to get right, and highly personal.\nWhat follows is my current favorite, and the journey to here.\nI'll try to note cases where I'm aware of my own subjectivity.\nbut I probably won't get everything.
\nI've worn headphones for a significant chunk of my computer time (an inordinate portion of my life) since 2011 or so.\nThat's about 15 years at the time of this writing.\nHere are some things that are important for me (or not important), which may help frame my opinion:
\nOver the years, I've really liked Sennheiser.\nI had several mid to high end pairs over the years.\nI rather liked my HD558s from years past.\nThey were mostly neutral response and open back.\nBut those were a long time ago (I bought them in 2013).\nAnd current Sennheiser isn't what they once were.
\nMore recently, I bought the Grado 325x.\nThey were crisp on the high end.\nBut after about 8 months, I couldn't stand them any more.\nThery were far too unforgiving of all but a perfectly mastered track.\nThe lower frequencies were off, especially for electronic music (but sounded great for rock).\nThey were heavy.\nThey were uncomfortable (I hated the headband and the weird angle of the earpieces,\nwhich look cool as a piece of steampunk concept art but are highly impractcial IMO).
\nTo make matters worse, the cable is non-detatchable and soldered all the way through.\nIt also weighs a ton and is super inflexible.\nI hear there are a bunch of people that do cable mods,\nbut by the time my pair developed a loose connection (probably due to the above)\nthat made them practically unlistenable, I was ready for something else.
\nModerately annoyed, I decided to buy some new headphones almost a year ago.\nI didn't find much help in reviews honestly, and you probably shouldn't expect to here either.\nBut I ended up going for some Beyerdynamic DT 900 Pro X's.\nI definitely over-indexed on the detatchable cable by the way\n(I haven't had a single issue with the cabling, nor have I bothered detaching it, but it's the principle of the thing).\nThe sound quality on these is excellent, and they have a better sound stage than my old Sennheiser's.\nI liked some things about the Grados better, but these are way more versatile.\nThey are also comfortable to wear for 2-3 hours:\nthey are lighter, and the cushioning is amazing.\nEspecially with my glasses.\nMy only complaint is they are a bit tight fitting for my large head,\nand I can actually hear some squeaks from my glasses frame when moving around sometimes.
\nHeadphones also need a solid amp (+ DAC if you're using it like most people).\nI currently use a Schiit Magni.\nA ridiculously under-priced amp+DAC.\nI really don't have much to say except that it's an excellent little unit that sounds great.\nI haven't had any issues in the 2 years I've owned it,\nbeyond the decade and change issues with certain music players not filling buffers fast enough\nduring a cold start of a new album.
\n", "summary": "", "date_published": "2025-11-25T00:00:00-00:00", "image": "", "authors": [ { "name": "Ian Wagner", "url": "https://fosstodon.org/@ianthetechie", "avatar": "media/avi.jpeg" } ], "tags": [ "gear" ], "language": "en" }, { "id": "https://ianwwagner.com//elecom-deft-pro-two-months-in.html", "url": "https://ianwwagner.com//elecom-deft-pro-two-months-in.html", "title": "Elecom Deft Pro - Two Months In", "content_html": "It's not quite two months in, but I said I'd give an update on the Elecom Deft Pro after some use, so here I am!
\nBad news first...
\nI'm not gonna lie, the initial adjustent period on this was rougher than I expected.\nThis trackballl was extremely uncomfortable for the first week,\nand not particularly comfortable for the following 2 or 3 weeks either.\nExpect to take some time getting used to this.
\nMy biggest complaint is probably that it's too small and too light.\nIt should be 2-3 times heavier in my opinion to stay put.\nAnd I guess I should have guessed it from the name, but it's really small.\nAlmost too small for my hands.
\nI'm not sure what it is with Elecom, but a lot of the angles make no sense to me.\n"Straight" as measured from the USB port puts the angle for the main ("left") click button off by like 5-10 degrees.\nSo I always want to have it off center, but that's challenging on my admittedly small keyboard tray.\nI'm also not really sure whose hand they designed whatever is further right than the right (alt-right?) button for.\nI never planned on using this or the middle click button anyways, but they are just weirdly positioned.\nThe forward and back buttons are also weirdly positioned.
\nThese were also weirdly positioned on the HUGE, but the Deft Pro seems worse.\nIt's exacerbated by the fact that the thing weighs basically nothing,\nwhich is not a positive thing when you're trying to poke it at weird angles that will definitely move it on your desk.\nAnd I'm not sure what the other weird button and switch are closest to the hand rest and on level with the left click button,\nbut they are in such weird positions I will NEVER use them.
\nOh my, this was a huge (lol) adjustment...\nThe scroll wheel is SO CRISP!\nI also kinda hate it, since even at max speed it's way too slow.\nI literally reach for my MacBook trackpad any time I want to scroll fast through a large document.\nThe scroll wheel is precise, but it's also kinda terrible IMO.\nAnd unlike the HUGE where you could sort of scroll horizontally by awkwardly nudging the wheel up or down,\nthe Deft is so tiny and light, and the wheel is so stiff, that you can just forget even the most awkward horizontal scrolling.\nUse your laptop trackpad for this.
\nI'm sure I sound VERY negative at this point.\nAnd I really do have a lot of bad to say about this trackball.\nBut I'm still using it and actually do have some good things to say.
\nHonestly I don't have any complaints about this.\nI saw some very polarizing opinions about the ball not rolling smoothly but I have had relatively few issues.\nIt's not sticking or anything for me and feels pretty good overall.\nMy precision feels slightly better than with the HUGE.
\nThe reason I replaced my HUGE was the unreliable buttons.\nMaybe my HUGE was a lemon, but the Deft Pro has been great.\nNo missed / duplicate click events.\nIt really just works.
\nWell, I'm still using it ;)\nSeveral things annoy me, but it's better than my HUGE.\nThe other trackball I considered was a Kensigton.\nI might still buy one, but only after I either:
\nOverall it's not a bad trackball, and you might even like it if you're primarily working mobile.\nBut I use this at my large desk and I have a few complaints.\nDespite my ergonomic complaints, it's still better than most mice,\nand your experience will probably be better than mine since my hands are large enough to make serious piano players envious.
\n", "summary": "", "date_published": "2025-11-19T00:00:00-00:00", "image": "", "authors": [ { "name": "Ian Wagner", "url": "https://fosstodon.org/@ianthetechie", "avatar": "media/avi.jpeg" } ], "tags": [ "gear" ], "language": "en" }, { "id": "https://ianwwagner.com//korean-music-beyond-k-pop-demon-hunters.html", "url": "https://ianwwagner.com//korean-music-beyond-k-pop-demon-hunters.html", "title": "Korean Music Beyond K-Pop Demon Hunters", "content_html": "Pretty much overnight, it seems that a movie I had never even heard of became the most-watched movie ever on Netflix.\nPeople everywhere are now singing the songs from K-Pop Demon Hunters.\nIt originally seemed more popular outside Korea\n(the first dozen or so people I heard about it from were not Korean, nor were they living here).\nBut now it's all over the promotional marketing locally too.
\nI absolutely love living here,\nand am immersed in many aspects of the culture,\nbut have never been a big K-pop fan.\nIn fact, I don't actually like most pop music anywhere in the world ;)
\nIf you, like me, have different musical tastes from the mainstream,\nthen this post is for you!\nIt's a collection of some of my favorite Korean music\nthat you've probably never heard of.
\nThere is no particular order to this post, so don't take this as some sort of ranking.\nIt's just a list of music and artists I like which can broaden your horizons.
\nOkay, let's kick off with some light... hip-hop?\nI don't really even know how to categorize this pair of songs,\nseparated by almost a decade but having a lot of similarities in both style and name.\nBefore the current American-influenced styles of rap and hip-hop started to dominate,\nthere was this more fun and weird side.
\nFirst up is boy band H.O.T. with Candy.\nThis was 1996, so it's probably the oldest on my list.\nIt's just light and fun.\nAnd very 90s... lighter side of hip hop.
\nOver the next decade, hip-hop and rap grew even more popular.\nThis next song, Lollipop, has one of the more bizarre music videos,\nand is a collaboration with more people than you'll be able to keep track of.\nHere's the MV.
\nThis song was released in the year 2000.\nThis style of dance+rock fusion was actually quite popular in Korea at the time,\nbut you don't hear very many producers putting out stuff like this today.\nThis song is well-known and loved by many in their late 30s or older,\nand is one of the most popular 노래방 (singing room = Karaoke) songs from that era,\neven today.\nThe song is full of passion and energy,\nand is very technically challenging.\nHere's a recent live performance.
\nThe gist of the song is a post-breakup story.\nIt's more of a power ballad, which is NOT my usual style of music.\nBut this song is just SO good, it has to be on my list.\nWhen it was released in 2017, it did extremely well,\nand got loads of radio play, despite not fitting the typical pop mold of the time.\nHere's a live performance\nthat also has some English subtitles.
\nOkay, this song probably does qualify as K-pop...\nThere is some good stuff in there ;)
\nThis is also the newest release on the list.\nIt grabbed my attention immediately\nbecause the musical structure, choice of scales, and rhythms\naren't typical in current K-pop.
\nHere's the music video.\nIt's definitely an idol group,\nbut the MV is artistically solid and generally quite creative\n(it alludes to both streamer culture and Korean-style photo booths)!
\nI have a hard time recommending a specific track,\nbecause this guy is a pretty wide ranging artist,\nand much of what he does is live.\nHe does everything from deep house on one of my favorite labels\nto ambient sets like this with a Eurorack modular.
\nThis song is getting REALLY out of the mold now...\nWe're in metal territory.
\nThe original was released back in 1997 by the group N.E.X.T,\nbut a newer in 2016 on a Korean TV show "The King of Mask Singer" made the song much more popular.\n(The concept of the show is that some famous singer puts on a mask, sings a song,\nand then celebrity judges have to guess who's behind the mask at the end).
\nOn one of these episodes, 하현우 does an absolutely stellar cover,\nwhich I liked even better than the original.\nHis vocal style is really on point for the song,\nand the high notes are absolutely killer!
\nYou can watch a video of the performance on YouTube.\nIt's truly something with a full orchestra, drums, guitar... the whole bit.
\nLast but definitely not least,\nthis song is fun, upbeat, and a bit punk-y in style.\nIt was released back in 2003 and bears a lot of resemblance to J-Rock at the time.
\nThe song is about a duck who dreams he can fly.\nHis mom scolds him and says ducks can't fly,\nbut that doesn't stop him from dreaming.
\nIt's such a beautiful song!\nAnd I love the metaphor about chasing your dreams.\nSomething that I really hope can continue to inspire the younger generation.
\nAnd the vocal style of the lead singer is pretty unique.\nShe's happy and has piercing high notes, but also has that rock edge at several points.\nAnd in live performances\nshe does this cute little hand motion throughout,\nmimicking wing flapping motion, which I thought was a cool and endearing!
\nI had this song stuck in my head for weeks after I discovered it.
\n", "summary": "", "date_published": "2025-11-09T00:00:00-00:00", "image": "", "authors": [ { "name": "Ian Wagner", "url": "https://fosstodon.org/@ianthetechie", "avatar": "media/avi.jpeg" } ], "tags": [ "music", "culture", "korea" ], "language": "en" }, { "id": "https://ianwwagner.com//const-assertions.html", "url": "https://ianwwagner.com//const-assertions.html", "title": "Const Assertions", "content_html": "I'm currently working on a project which involves a lot of lower level\ndata structures.\nBy lower level I mean things like layout and bit positions, and exact sizes being important.\nAs such, I have a number of pedantic lints enabled.
\nOne of the lints I use is cast_precision_loss.\nFor example, casting from usize to f32 using the as keyword is not guaranteed to be exact,\nsince f32 can only precisely represent integrals up to 23 bits of precision (due to how floating point is represented).\nAbove this you can have precision loss.
This lint is pedantic because it can generate false positives where you know the input can't ever exceed some threshold.\nBut wouldn't it be nice if we could go from "knowing" we can safely disable a lint to actually proving it?
\nThe first thing that came to mind was runtime assertions, but this is kind of ugly.\nIt requires that we actually exercise the code at runtime, for one.\nWe should be able to cover this in unit tests, but even if we do that,\nan assertion isn't as good as a compile time guarantee.
\nconstOne thing I didn't mention, and the reason I "know" that the lint would be fine is that I'm using a const declaration.\nHere's a look at what that's like:
pub const BUCKET_SIZE_MINUTES: u32 = 5;\npub const BUCKETS_PER_WEEK: usize = (7 * 24 * 60) as usize / BUCKET_SIZE_MINUTES as usize;\nThis isn't the same as a static or a let binding.\nconst expressions are actually evaluated at compile time.\n(Well, most of the time... there's a funny edge case where const blocks which can never executed at runtime\nis not guaranteed to evaluate.)
You can't do everything in const contexts, but you can do quite a lot, including many kinds of math.\nNot all math; some things like square root and trigonometry are not yet usable in const contexts\nsince they are not reproducible across architectures (and sometimes even on the same machine, it seems).
assert! in a const blockAnd now for the cool trick!\nI want to do a division here, and to do so, I need to ensure the types match.\nThis involves casting a usize to f32,\nwhich can cause truncation as noted above.
But since BUCKETS_PER_WEEK is a constant value,\nwe can actually do an assertion against it in our const context.\nThis lets us safely enable the lint, while ensuring we'll get a compile-time error if this ever changes!\nThis has no runtime overhead.
#[allow(clippy::cast_precision_loss, reason = "BUCKETS_PER_WEEK is always <= 23 bits")]\nconst PI_BUCKET_CONST: f32 = {\n // Asserts the invariant; panics at compile time if violated\n assert!(BUCKETS_PER_WEEK < 2usize.pow(24));\n\t// Computes the value\n std::f32::consts::PI / BUCKETS_PER_WEEK as f32\n};\nThis is all possible in stable Rust at the time of this writing (tested on 1.89).\nI saw some older crates out there which appeared to do this,\nbut as far as I can tell, they are no longer necessary.
\nHere's a Rust Playground\npreloaded with the sample code\nwhere you can verify that changing BUCKETS_PER_WEEK to a disallowed value causes a compile-time error.
I recently picked up a new trackball to replace my prematurely aging Elecom HUGE.\nHere are my first impressions after only a few hours, since there wasn't much I could find online comparing them properly.
\nNOTE: I have an updated review after ~2 months with the unit here.
\nTo get the obvious question out of the way first,\nI have used a trackball for about 5 years as my primary pointer device.\nI switched from all manner of mice and trackpads due to ergonomics.\nI had pretty bad RSI for many years, due to excessive computer use,\nand a trackball was one of the key changes that solved my issues.
\nThe trackball actually had a larger impact than switching to Dvorak (minimal improvement)\nor switching to a better ergonomic keyboard (Kinesis Advantage 2 QD; incremental, but not order of magnitude improvement).\nAnd yes, I'm aware of vertical mice.\nI tried a Logitech one for about 2 years and it actually made things worse.
\nThe biggest difference is probably the size.\nThe Deft Pro has a much smaller footprint than the HUGE.\nPart of the reason for this early replacement is that I ordered a replacement keyboard drawer online the other week.\nBeing too lazy to measure, and just going off the scale photos, it looked plenty big.
\n
Narrator: it was, in fact, far too small and barely fits the keyboard.
\nSo, with the HUGE practically falling off,\ncombined with my bad habit of snacking and drinking at my desk late at night contributing to click malfunction\n(weirdly it would DOUBLE register clicks!), I needed a replacement.
\n![\"An](\"media/deft-on-the-tray.jpeg\")
The Deft Pro is pretty much the size I wanted.\nIt fits (just barely) on my tiny keyboard tray,\nand it is quite comfortable to use.
\nI have to say I wish this thing were a bit heavier.\nI want the thing to sit on my desk like a rock and not move.\nI tend to manipulate the ball with my fingertips,\nand keep other fingers on the buttons, but this tends to push the base off to the left.\nMaybe I'll look at a sticky setup or something...
\nI can only give a partial review here, since I do not use most of the extra buttons.\nPartly because I can't be bothered to configure them (most apps won't support them easily),\nand partly because I don't like installing proprietary software with low level system access for marginal benefit.\n(This is a subtoot at Logitech Options, but Elecom is not much better.)\nSo, I only use the left+right click, and the forward/back buttons.\nThe forward/back buttons are plug-and-play\nwith my web browser and many dev tools like JetBrains IDEs.
\nI think the HUGE positioning was a bit better for the forward/back buttons.\nThe scroll and left click position are fine for my usage style.\nThe right click though... That will take some getting used to.\nI think they designed the Deft to be held in your hand pretty closely,\nbut that isn't how I got used to using the HUGE.
\nI'll give this a shot and see how it goes after a few weeks trying a different grip.\nI think right click is positioned fine for the intended style of gripping.
\nThe Deft allows for much better precision, with each step being a nice crisp click.\nI personally liked the HUGE better.\nIt was pretty fluid, smooth, and not "clicky."\nIt also could generate acceptable scrolling speeds.\nNot as good as an Apple trackpad, but acceptable.\nI had to jack up the scroll speed to max for the Deft,\nand will probably continue using my trackpad for gestures like this when I want to go fast.
\nThis is not a criticism of Elecom specifically; it's a universal gripe I have with most things\nexcept the Apple trackpad.
\nThe ball itself is great!\nI found some reviews online complaining that it was sticky,\nor hard to manipulate precisely.\nI have not had any issues so far; it feels as good or better than the HUGE.
\nIf I recall, the HUGE required you to select a connectivity option at purchase time.\nThe Deft Pro includes all in the same package: bluetooth, RF via USB wireless dongle, and cabled USB connection.\nThis is great if you like to travel with an external input device!
\nI personally don't since I love my MacBook trackpad.\nBut if you do, the only thing I'd flag is that there is no internal battery.\nYou need a single AA battery rather than an internal rechargeable like the higher end Logitech mice.
\nIt lives up to the name: it's Deft.\nI like a few things about the HUGE better (the scroll wheel and right mouse button position being the main ones),\nbut I did buy this for a smaller footprint,\nand I think I can get used to the button position with a tighter grip.
\nI'll post a follow up later once I have a chance to use it more.
\n", "summary": "", "date_published": "2025-09-28T00:00:00-00:00", "image": "media/huge-on-the-tray.jpeg", "authors": [ { "name": "Ian Wagner", "url": "https://fosstodon.org/@ianthetechie", "avatar": "media/avi.jpeg" } ], "tags": [ "gear" ], "language": "en" }, { "id": "https://ianwwagner.com//finishing-dragonball-in-korean.html", "url": "https://ianwwagner.com//finishing-dragonball-in-korean.html", "title": "Finishing Dragonball in Korean", "content_html": "Over the past few days of vacation I finally had the chance to finish reading Dragonball!\nI started this a few years ago and have gone in spurts, sometimes not reading for months at a time as work got in the way.
\nThis post is a short reflection on what I learned,\nwhich I hope will be useful for other language learners.
\nI did this because I wanted to improve my Korean for various practical reasons,\nbut I'm also very lazy when it comes to studying.\nI also tend to like Japanese media a bit better than Korean\n(a few Korean webtoons and comics are good, but I'm more a fan of the Japanese manga art style).\nSo I split the difference and tried to find manga that had high quality Korean translations.\nI also have fond memories of devouring comics as a kid (mostly Calvin & Hobbes),\nand am sure that did something to improve my reading.
\nWith the general medium and genre decided, I looked for a series.\nI had watched several of the classics in anime form, including One Piece and Fairy Tail,\nbut they were a bit too advanced for my Korean level ~4 years ago when I started this.\nThen an acquaintance who is fluent in Japanese mentioned that he had read Dragonball in Japanese\nmultiple times.
\nI took a look at the preview and was surprised to find that it was not TOO far above my own level.\nIt was definitely challenging, but I could understand something like half the words,\nand most of the grammatical constructions, which was something!\nI should also not that I had never read any of the English translation,\nnor had I watched any of the English dubbed anime / films.
\nI was extremely lazy the first few years, and it was difficult to get going.\nReading was also a bit demoralizing at first with a failure rate of something like 50-60% for a while.\nBut the vocabulary and patterns were pretty repetitive so I did improve pretty quickly.
\nI found it helpful to force myself to NOT use a dictionary after the first few volumes.\nThis is how most kids learn a language anyways, and it had a few benefits for me:
\nThe story was (IMO) significantly more engaging for the first 16 volumes,\nand the earlier chapters also had more attention to artwork detail in many respects.\nI don't regret reading all the way to the end, but it got quite repetitive and predictable to be honest.
\nTo complement the reading (and provide a reward for the slog initially),\nI also watched Dragonball and Dragonball Z Kai on Laftel (NOTE: This may be region blocked),\nreading the chapters BEFORE watching them most of the time.\nI think that helped me focus on the text and improve my (weaker) reading.
\nI significantly improved my Korean reading comprehension and vocabulary.\nI also improved my knowledge of more advanced grammatical structures,\nbut to a much lesser extent.\nDragonball is extremely repetitive, which was great for reinforcement learning,\nbut it didn't dramatically expand my grammatical prowess.
\nOne unexpected area I also improved was my ability to understand humor and make jokes!\nEspecially word plays.\nDragonball's Korean translation is excellent, and given the cultural similarity,\na lot of the subtler jokes around things like age and respect translate well.
\nI can also sound like a cheeky teenager and know a lot of ridiculous sounding insults!
\nThis will NOT help so much if you're interested primarily in learning household / everyday conversational patterns,\nbut it IS conversational, so it can be a good complement to other TV shows and books.
\nWhat will I do next?\nI haven't really decided on the next series,\nbut One Piece and Fairy Tail are probably reasonable options now.
\nI think I might also work on my Japanese a bit on the side.\nMy Japanese is nowhere near as good as my Korean,\nbut the success of this experiment makes me think I could probably work my way slowly through the first 16 volumes.\nAs it's somewhat aimed at kids, it helpfully has Furigana so I could really level up my Kanji recognition.
\n", "summary": "", "date_published": "2025-09-08T00:00:00-00:00", "image": "", "authors": [ { "name": "Ian Wagner", "url": "https://fosstodon.org/@ianthetechie", "avatar": "media/avi.jpeg" } ], "tags": [ "learning", "languages", "self-improvement" ], "language": "en" }, { "id": "https://ianwwagner.com//unzip-utf-8-docker-and-c-locales.html", "url": "https://ianwwagner.com//unzip-utf-8-docker-and-c-locales.html", "title": "Unzip, UTF-8, Docker, and C Locales", "content_html": "Today's episode of "things that make you go 'wat'" is sponsored by unzip.\nYes, the venerable utility ubiquitous on UNIX-like systems.\nI mean, what could possibly go wrong?
This morning I was minding my own business sipping a coffee,\nwhen suddenly the "simple" CI pipeline I was working on failed.\nI literally copied the command that failed out of a shell script.\nWhich ran on the exact same machine previously.\nAnd the command that failed was a simple unzip.\nHuh?
I initially assumed the file may have been corrupted, or maybe the archive had failed in a weird way\n(I don't control the process, and was downloading it from the internet).\nWhile it was running again, I started hitting Page Down looking for oddities in the log.\nI was greeted by this near the end of the output:
se/municipality_of_savsj#U00f6-addresses-city.geojson: mismatching "local" filename (se/municipality_of_savsjö-addresses-city.geojson),\n continuing with "central" filename version\nWeird, huh?\nIt looks like it's trying to unzip a file with a UTF-8 name, which should be supported.
\nI ssh'd into the remote machine to give it a try in the terminal.\nI've run this command dozens of times, and wanted to see if I got the same output in a standard bash remote terminal.\nGiven that log output is sometimes inscrutable, I thought "maybe it wasn't the unzip itself that failed directly?"\nOr something...
Well, the unzip worked in the regular bash login shell on the same host.\nSo there must be something different about the CI environment.
As a first resort, I started digging through the help and man pages.\nThe first amusing factoid I learned was that apparently the last official release of this utility happened in 2009.\nI guess zip doesn't evolve much, eh?
\nThe man page was actually pretty detailed,\nbut it didn't have a lot to say about Unicode, or anything else obvious related to the error.\nAnd the project doesn't exactly have an active central issue tracker (there is one on SourceForge, but replies are few and far between).\nI couldn't find anyone else talking about this specific error in the usual places on the internet either.
\nI did however find this interesting bug report via a Debian mailing list\nfrom 2019.\nThe thing that caught my eye was the code sample and mention of locales.
\nOh no...\nWhat if this actually changes behavior based on your system locale?\nThen it started coming back to me... something about a dozen or so environment variables\nthat determine the behavior of C programs in bizarre ways...
\nSo I ran locale on the ssh session and got something reasonable back:
$ locale\nLANG=C.UTF-8\nLANGUAGE=\nLC_CTYPE="C.UTF-8"\nLC_NUMERIC="C.UTF-8"\nLC_TIME="C.UTF-8"\nLC_COLLATE="C.UTF-8"\nLC_MONETARY="C.UTF-8"\nLC_MESSAGES="C.UTF-8"\nLC_PAPER="C.UTF-8"\nLC_NAME="C.UTF-8"\nLC_ADDRESS="C.UTF-8"\nLC_TELEPHONE="C.UTF-8"\nLC_MEASUREMENT="C.UTF-8"\nLC_IDENTIFICATION="C.UTF-8"\nLC_ALL=\nThen, to avoid wasting even more hours of CI time (it's a long job whose source is a ~100GB ZIP file),\nI set about replicating the environment of the runner.\nFortunately it left a Docker volume behind for me.\nSo, I launched an interactive Docker container using the same image that the CI job used (python:3.13).\nSomewhat surprisingly, the output of locale was a bunch of variables set to.... nothing!\nNow we're getting somewhere!
To confirm that I could indeed reproduce the failure, I ran unzip again in the container,\nand sure enough, I got the same log message, and the exit code was 1.\nNow we've confirmed how to reproduce the issue at least, so we can test a fix.
unzipSince unzip is single threaded (read: SLOW),\nI spent my time during the tests looking through the source code to try and confirm my theory about the locale variables being the issue.\nThe official version seems to be hosted on SourceForge, which is apparently still a thing.\n(I'm sure there are a lot of Debian patches, but I just wanted a quick way to peruse the code).\nAmusingly, there were 47,074 downloads/week of unzip60.tar.gz, and only 36 downloads/week of the ZIP version.
Eventually, I found what I was looking for in unzip.c:
int unzip(__G__ argc, argv)\n __GDEF\n int argc;\n char *argv[];\n{\n#ifndef NO_ZIPINFO\n char *p;\n#endif\n#if (defined(DOS_FLX_H68_NLM_OS2_W32) || !defined(SFX))\n int i;\n#endif\n int retcode, error=FALSE;\n#ifndef NO_EXCEPT_SIGNALS\n#ifdef REENTRANT\n savsigs_info *oldsighandlers = NULL;\n# define SET_SIGHANDLER(sigtype, newsighandler) \\\n if ((retcode = setsignalhandler(__G__ &oldsighandlers, (sigtype), \\\n (newsighandler))) > PK_WARN) \\\n goto cleanup_and_exit\n#else\n# define SET_SIGHANDLER(sigtype, newsighandler) \\\n signal((sigtype), (newsighandler))\n#endif\n#endif /* NO_EXCEPT_SIGNALS */\n\n /* initialize international char support to the current environment */\n SETLOCALE(LC_CTYPE, "");\n\n#ifdef UNICODE_SUPPORT\n /* see if can use UTF-8 Unicode locale */\n# ifdef UTF8_MAYBE_NATIVE\n {\n char *codeset;\n# if !(defined(NO_NL_LANGINFO) || defined(NO_LANGINFO_H))\n /* get the codeset (character set encoding) currently used */\n# include <langinfo.h>\n\n codeset = nl_langinfo(CODESET);\n# else /* NO_NL_LANGINFO || NO_LANGINFO_H */\n /* query the current locale setting for character classification */\n codeset = setlocale(LC_CTYPE, NULL);\n if (codeset != NULL) {\n /* extract the codeset portion of the locale name */\n codeset = strchr(codeset, '.');\n if (codeset != NULL) ++codeset;\n }\n# endif /* ?(NO_NL_LANGINFO || NO_LANGINFO_H) */\n /* is the current codeset UTF-8 ? */\n if ((codeset != NULL) && (strcmp(codeset, "UTF-8") == 0)) {\n /* successfully found UTF-8 char coding */\n G.native_is_utf8 = TRUE;\n } else {\n /* Current codeset is not UTF-8 or cannot be determined. */\n G.native_is_utf8 = FALSE;\n }\n /* Note: At least for UnZip, trying to change the process codeset to\n * UTF-8 does not work. For the example Linux setup of the\n * UnZip maintainer, a successful switch to "en-US.UTF-8"\n * resulted in garbage display of all non-basic ASCII characters.\n */\n }\n# endif /* UTF8_MAYBE_NATIVE */\nAnd there we have it, right at the start of the program...
\nThis is a bit rough to grok if you don't regularly read C,\nbut the gist of it is that it tries to look at the system locale\nusing setlocale.\nThe LC_CTYPE specifies the types of character used in the locale.\nThe second argument is... very C.\nThe function will "install the specified system locale... as the new C locale."
It defaults to "C" at startup, per the docs.\nIn unzip.c though, the authors use the magic value "",\nwhose behavior is to set the locale to the "user-preferred locale".\nThis comes from those environment variables.\nA few lines down, past the #ifdefs gating unicode support,\nthey parse the codeset portion by passing NULL, which is ANOTHER magic value\nthat simply returns the current value.\n(So this two-step dance loads the preferred locale from the environment variables,\nwhich are empty in this case, and then inspects the codeset.).
If your environment variables do not explicitly specify UTF-8,\nyou will get some.... strange and undesirable behavior, apparently,\nif your archive contains unicode file names!
\nI'm pretty sure there's a good historical reason for this.\nGiven that many C library functions are locale-sensitive,\nand UTF-8 support is much younger than UNIX and (obviously) C.\nBut I found this behavior from unzip to be a bit surprising nonetheless.\nAnd rather than setting a sensible default like C.UTF-8,\nit turns out most Docker containers start with none!
Fortunately this is pretty easy to fix.\nJust export LC_ALL="C.UTF-8" before using unzip.\n(There is probably a more granular approach, but the LC_ALL sledgehammer does the job.)\nKinda crazy that you have to do this,\nbut hopefully this saves someone else an afternoon of debugging!
Forgive me in advance, but this post will probably be a bit rant-y.\nIf you're looking for a way to do container builds in GitLab CI without a lot of fuss,\nthis article is for you.
\nI'm writing this post because Google recently canned yet another project: Kaniko.\nThis used to be pretty much the only way to build container images in Kubernetes.\nThis probably hasn't been the case for quite some time,\nbut devops is a chore for me, and my level of involvement is usually\njust enough to get my actual work done.
\nAnyways, there are a few possible replacements, including podman buildah and BuildKit.\nWith not so much as a finger to the wind, I decided that BuildKit looked like the more polished and capable option,\nso I went with that (even though I actually use podman on servers way more often than Docker proper).\nI found the documentation for switching to be a bit lacking,\nso you all get this post!
\nI've used GitLab CI for years and find it to be an extremely capable and easy to configure system.\nRunners are extremely easy to set up.\nRead the docs for details, but I'll whizz through a Docker-based setup (I haven't gotten around to k8s, so this is plain Docker).
\nFirst, create a volume to persist the configuration:
\ndocker volume create buildkit-gitlab-runner-config\nThen, start the runner for the first time.\nI've added a few flags to set up the required volumes, and ensure it restarts automatically.
\ndocker run -d --name buildkit-gitlab-runner --restart always -v /var/run/docker.sock:/var/run/docker.sock -v buildkit-gitlab-runner-config:/etc/gitlab-runner gitlab/gitlab-runner:latest\nThe runner will launch, ish, but won't do anything useful without configuration + registration.\nYou can use an ephemeral instance of the runner image to register it.\nThis prompts you for your GitLab server URL and a token.\nYou get a token by logging into the GitLab admin section and registering a new runner.\nI also set the default image to moby/buildkit:rootless, but this is optional.
docker run --rm -it -v buildkit-gitlab-runner-config:/etc/gitlab-runner gitlab/gitlab-runner:latest register\nNext, get the path to the volume with the config using docker volume inspect buildkit-gitlab-runner-config.\nInside that directory (probably only readable by root),\nyou'll see the newly generated config.toml.
To enable running container builds, you'll need this in a runners.docker section:
\nsecurity_opt = ["seccomp:unconfined", "apparmor:unconfined"]\nThis is a bit confusing, since the GitLab docs do document this but,\nat the time of this writing, there are no examples and the description of the format is confusing.\nSee the BuildKit docs\nsection on Docker for an explanation of why these options are necessary but safe enough for rootless builds.\nThere's also a third flag, systempaths=unconfined, which I've omitted (we'll revisit in a moment).
In short, the above gets you rootless image builds from within a dockerized GitLab CI runner,\nwithout resorting to privileged containers or docker-in-docker.
\n.gitlab-ci.ymlNow let's look at what you need to get this working in your CI configuration.\nI'll assume your repo has a Dockerfile in it already and you want to build + push to your GitLab Container Registry.
\nstages:\n - build\n\ncontainerize:\n stage: build\n image:\n name: moby/buildkit:rootless\n entrypoint: [ "" ] # !!\n variables:\n BUILDKITD_FLAGS: --oci-worker-no-process-sandbox\n tags:\n - docker\n - buildkit-rootless\n before_script:\n # We have some more elaborate logic to our tag naming, but that's irrelevant...\n - export IMAGE_TAG="$CI_COMMIT_TAG"\n # Container registry credentials\n - mkdir -p ~/.docker\n - echo "{\\"auths\\":{\\"$CI_REGISTRY\\":{\\"username\\":\\"$CI_REGISTRY_USER\\",\\"password\\":\\"$CI_REGISTRY_PASSWORD\\"}}}" > ~/.docker/config.json\n script:\n - buildctl-daemonless.sh build\n --frontend dockerfile.v0\n --local context=.\n --local dockerfile=.\n --output type=image,name=$CI_REGISTRY_IMAGE:$IMAGE_TAG,push=true\n --opt build-arg:CI_JOB_TOKEN=$CI_JOB_TOKEN\nThis should be pretty much copy+paste for most projects.\nA few things to note:
\nBUILDKITD_FLAGS using the flag that BuildKit discourages.\nSince we'd like the same config to work on k8s runners too, we have to use this.ARG. Why? I'm glad you asked...We have a lot of internal crates in private repos.\nRecently I hit a snag with our previous approach to authenticated pulls though.\nOur Cargo.toml files use git SSH links, which won't work in CI.\nBut you can authenticate using HTTPS and a job token!
Our previous approach was to use sed to rewrite Cargo.toml and Cargo.lock.\nIt worked until we had a transitive dependency (a direct dependency on one private crate,\nwhich in turn had a dependency on another private crate).\nI don't know why this broke exactly, since we do use Cargo.lock,\nbut regardless, it was brittle.
The solution was to amed our Dockerfile with an ARG CI_JOB_TOKEN.\nThe build script then does some git magic to rewrite the SSH requests into HTTPS ones.\nI don't know why this feature exists,\nbut I'm happy I don't need to figure out how to run a private crate registry!
# Git hacks\ngit config --global credential.helper store\necho "https://gitlab-ci-token:${CI_JOB_TOKEN}@git.mycompany.com" > ~/.git-credentials\ngit config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@git.mycompany.com".insteadOf ssh://git@git.mycompany.com\nJust add this to your build script and replace the domain with your actual GitLab domain,\nand you'll have no more issues with transitive dependencies and authenticated pulls.
\n", "summary": "", "date_published": "2025-07-31T00:00:00-00:00", "image": "", "authors": [ { "name": "Ian Wagner", "url": "https://fosstodon.org/@ianthetechie", "avatar": "media/avi.jpeg" } ], "tags": [ "devops" ], "language": "en" }, { "id": "https://ianwwagner.com//optimizing-rust-builds-with-target-flags.html", "url": "https://ianwwagner.com//optimizing-rust-builds-with-target-flags.html", "title": "Optimizing Rust Builds with Target Flags", "content_html": "Recently I've been doing some work using Apache DataFusion for some high-throughput data pipelines.\nOne of the interesting things I noticed on the user guide was the suggestion to set\nRUSTFLAGS='-C target-cpu=native'.\nThis is actually a pretty common optimization (which I periodically forget about and rediscover),\nso I thought I'd do a quick writeup on this.
A compiler translates your "idiomatic" code into low-level instructions.\nModern optimizing compilers are pretty good at figuring out ways to cleverly rewrite your code\nto make it faster, while still being functionally equivalent at execution time.\nThe instructions may be reordered from what your simple mental model expects,\nand they may even have no resemblance.\nThis includes rewriting some loop-like (or iterator) patterns into "vectorized" code using SIMD instructions\nthat perform some operation on multiple values at once.
\nSpecial instruction families like this often vary within a single architecture,\nwhich may be surprising at first.\nThe compiler can be configured to enable (or disable!) specific "features",\noptimizing for compatibility or speed.
\nIn rustc, each target triple has a default set of CPU features enabled.\nIn the case of my work laptop, that's aarch64-apple-darwin.\nSince this architecture doesn't have a lot of variation among chips,\nthe compiler can make some pretty good assumptions about what's available.\n(In fact, for my specific CPU, the M1 Max, it's perfect!)\nBut we'll soon see this is not the case for the most common target: x86_64 Linux.
To figure out what features we could theoretically enable,\nwe need some CPU info from the machine we intend to deploy on.\nThe canonical way of checking CPU features on Linux is probably to cat /proc/cpuinfo.\nThis gives a lot more output that you probably need though.\nHelpfully, rustc includes a simple command that shows you the config\nfor the native CPU capabilities: rustc --print=cfg -C target-cpu=native.\nHere's what it looks like on one Linux machine:
debug_assertions\npanic="unwind"\ntarget_abi=""\ntarget_arch="x86_64"\ntarget_endian="little"\ntarget_env="gnu"\ntarget_family="unix"\ntarget_feature="adx"\ntarget_feature="aes"\ntarget_feature="avx"\ntarget_feature="avx2"\ntarget_feature="bmi1"\ntarget_feature="bmi2"\ntarget_feature="cmpxchg16b"\ntarget_feature="f16c"\ntarget_feature="fma"\ntarget_feature="fxsr"\ntarget_feature="lzcnt"\ntarget_feature="movbe"\ntarget_feature="pclmulqdq"\ntarget_feature="popcnt"\ntarget_feature="rdrand"\ntarget_feature="rdseed"\ntarget_feature="sse"\ntarget_feature="sse2"\ntarget_feature="sse3"\ntarget_feature="sse4.1"\ntarget_feature="sse4.2"\ntarget_feature="ssse3"\ntarget_feature="xsave"\ntarget_feature="xsavec"\ntarget_feature="xsaveopt"\ntarget_feature="xsaves"\ntarget_has_atomic="16"\ntarget_has_atomic="32"\ntarget_has_atomic="64"\ntarget_has_atomic="8"\ntarget_has_atomic="ptr"\ntarget_os="linux"\ntarget_pointer_width="64"\ntarget_vendor="unknown"\nunix\nAside: I'm not quite sure why, but this isn't a 1:1 match with /proc/cpuinfo on this box!\nIt definitely does support some AVX512 instructions,\nbut those don't show up in the native CPU options.\nIf anyone knows why, let me know!
UPDATE: Shortly after publishing this post, Rust 1.89 was released.\nThis PR linked in the release notes caught my eye.\nApparently the target features for AVX512 were not actually stable at the time of writing,\nbut they are now.\nRe-running the above command with version 1.89 of rustc now includes the AVX512 instructions.
\nPerhaps the more interesting question which motivates this investigation is\nwhat the defaults are.\nYou can get this with rustc --print cfg.\nThis shows what you get when you run cargo build without any special configuration.\nHere's the output for the same machine:
debug_assertions\npanic="unwind"\ntarget_abi=""\ntarget_arch="x86_64"\ntarget_endian="little"\ntarget_env="gnu"\ntarget_family="unix"\ntarget_feature="fxsr"\ntarget_feature="sse"\ntarget_feature="sse2"\ntarget_has_atomic="16"\ntarget_has_atomic="32"\ntarget_has_atomic="64"\ntarget_has_atomic="8"\ntarget_has_atomic="ptr"\ntarget_os="linux"\ntarget_pointer_width="64"\ntarget_vendor="unknown"\nunix\nWell, that's disappointing, isn't it?\nBy default, you'd only get up to SSE2, which is over 20 years old by now!\nThis is a consequence of the diversity of the x86_64 architecture.\nIf you want your binary to run everywhere, this is the price you'd have to pay.
While -C target-cpu=native will usually make your code faster on the build machine,\na lot of modern software is built by a CI pipeline on cheap runners, but deployed elsewhere.\nTo reliably target a specific set of features, use the target-feature flag.\nThis lets you specifically enable features you know will be available on the machine running the code.\nHere's an example of RUSTFLAGS that incorporates all of the above features.\nThis should enable builds to proceed from any other x86_64 Linux machine and while producing a binary\nthat supports the exact features of the deployment machine.
RUSTFLAGS="-C target-feature=+adx,+aes,+avx,+avx2,+bmi1,+bmi2,+cmpxchg16b,+f16c,+fma,+fxsr,+lzcnt,+movbe,+pclmulqdq,+popcnt,+rdrand,+rdseed,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+xsave,+xsavec,+xsaveopt,+xsaves"\nA few days after writing this, I accidentally stumbled upon something else when working out target flags\nfor a program I knew would have wider support across several datacenters.\nIt sure would be nice if there were some "groups" of commonly supported features, right?
\nTurns out this exists, and it was staring right at me in the CPU list: microarchitecture levels!\nIf you list out all the available target CPUs via rustc --print target-cpus on a typical x86_64 Linux box,\nyou'll see that your default target CPU is x86-64.\nThis means it will run on all x86_64 CPUs, and as we discussed above, this doesn't give much of a baseline.\nBut there are 4 versions in total, going up to x86-64-v4.\nIt turns out that AMD, Intel, RedHat, and SUSE got together in 2020 to define these,\nand came up with some levels which are specifically designed for our use case of optimizing compilers!\nYou can find the full list of supported features by level on Wikipedia\n(search for "microarchitecture levels").
rustc --print target-cpus will also tell you which specific CPU you're on.\nYou can use this info to find which "level" you support.\nBut a more direct way to map to level support is to run /lib64/ld-linux-x86-64.so.2 --help.\nThanks, internet!\nYou'll get some output like this on a modern CPU:
Subdirectories of glibc-hwcaps directories, in priority order:\n x86-64-v4 (supported, searched)\n x86-64-v3 (supported, searched)\n x86-64-v2 (supported, searched)\nAnd if you run on slightly older hardware, you might get something like this:
\nSubdirectories of glibc-hwcaps directories, in priority order:\n x86-64-v4\n x86-64-v3 (supported, searched)\n x86-64-v2 (supported, searched)\nThis should help if you're trying to aim for broader distribution rather than enabling specific features for some known host.\nThe line to target an x86_64 microarch level is a lot shorter.\nFor example:
\nRUSTFLAGS="-C target-cpu=x86-64-v3"\nNOTE: As mentioned above, Rust 1.89 was released shortly after this post.\nThis incidentally brings support for AVX512 CPU features in the x86-64-v4 target CPU,\nwhich were previously marked unstable.
Enabling CPU features doesn't always make things faster.\nIn fact, in some cases, it can even do the opposite!\nThis thread\nhas some interesting anecdotes.
\nIn conclusion, here's a quick reference of the useful commands we covered:
\nrustc --print cfg - Shows the compiler configuration that your toolchain will use by default.rustc --print=cfg -C target-cpu=native - List the configuration if you were to specifically target for your CPU. Use this to see the delta between the defaults and the featurse supported for a specific CPU.rustc --print target-cpus - List all known target CPUs. This also tells you what your current CPU and what the default CPU is for your current toolchain./lib64/ld-linux-x86-64.so.2 --help - Specifically for x86_64 Linux users, will show you what microarchitecture levels your CPU supports.rustc --print target-features - List all available target features with a short description. You can scope to a specific CPU with -C target-cpu=. Useful mostly to see what you're missing, I guess.Rust's ownership system is well-known for the ways it enforces memory safety guaranteees.\nFor example, you can't use some value after it's been freed.\nFurther, it also ensures that mutability is explicit,\nand it enforces some extra rules that make most data races impossible.\nBut the ownership system has benefits beyond this which don't get as much press.
\nLet's look at a fairly common design pattern: the builder.\nA builder typically takes zero or few arguments to create.\nIn Rust, it's often implemented as a struct that implements the Default trait.\nThen, you progressively "chain" method invocations to ergonomically\nto specify how to build the thing you want.\nFor example:
let client = reqwest::Client::builder()\n .user_agent(APP_USER_AGENT)\n .timeout(Duration::from_secs(3))\n .build()?;\nThis pattern is useful because it avoids bloated constructors with dozens of arguments.\nIt also lets you encode failability into the process:\nsome combinations of arguments may be invalid.
\nIf you look at the signature for the timeout function,\nyou'll find it takes self as its first parameter and returns a value of the same type.\nThe key thing to note is that a non-reference self parameter\nwill "consume" the receiver!\nSince it takes ownership, you can't hold on to a reference to the original value!
This prevents a whole class of subtle bugs.\nPython, for example, doesn't prevent you from modifying inputs,\nand it's not always clear if a function/method is supposed to return a new value,\nwhether that value has the same contents as the original reference (which is still valid!)\nor if it's completely fresh,\nand so on.
\nA few other languages, mostly in the purely functional tradition (Haskell comes to mind)\nalso have a similar property.\nThey don't use a concept of "ownership" but rather remove mutability from the language.\nRust makes what I consider to be a nice compromise\nwhich retains most of the benefits while being easier to use.
\nIn summary, the borrow checker is a powerful ally,\nand you can leverage it to make truly better APIs,\nsaving hours of debugging in the future.
\n", "summary": "", "date_published": "2025-05-31T00:00:00-00:00", "image": "", "authors": [ { "name": "Ian Wagner", "url": "https://fosstodon.org/@ianthetechie", "avatar": "media/avi.jpeg" } ], "tags": [ "rust", "functional programming" ], "language": "en" }, { "id": "https://ianwwagner.com//unicode-normalization.html", "url": "https://ianwwagner.com//unicode-normalization.html", "title": "Unicode Normalization", "content_html": "Today I ran into an amusingly named place,\nthanks to some sharp eyes on the OpenStreetMap US Slack.\nThe name of this restaurant is listed as "𝐊𝐄𝐁𝐀𝐁 𝐊𝐈𝐍𝐆 𝐘𝐀𝐍𝐆𝐎𝐍".\nThat isn't some font trickery; it's a bunch of Unicode math symbols\ncleverly used to emphasize the name.\n(Amusingly, this does not actually show up properly on most maps, but that's another story for another post).
\nI was immediately curious how well the geocoder I spent the last few months building handles this.
\n![\"A](\"media/kebab-king-duplicates.png\")
Well, at least it found the place, despite the very SEO-unfriendly name!\nBut what's up with the second search result?
\nWell, that's a consequence of us pulling in data from multiple sources.\nIn this case, the second result comes from the Foursquare OS Places dataset.\nIt seems that either the Placemaker validators decided to clean this up,\nor the Foursquare user who added the place didn't have that key on their phone keyboard.
\nOne of the things our geocoder needs to do when combining results is deduplicating results.\n(Beyond that, it needs to decide which results to keep, but that's a much longer post!)\nWe use a bunch of factors to make that decision, but one of them is roughly\n"does this place have the same name," where same is a bit fuzzy.
\nOne of the ways we can do this is normalizing away things like punctuation and diacritics.\nThese are quite frequently inconsistent across datasets, so two nearby results with similar enough names\nare probably the same place.\nFortunately, Unicode provides a few standardized transformations into canonical forms\nthat make this easier.
\nWhat we think of as a "character" does not necessarily have a single representation in Unicode.\nFor example, there are multiple ways of encoding "서울" which will look the same when rendered,\nbut have a different binary representation.\nThe Korean writing system is perhaps a less familiar case for many,\nbut characters with diacritical marks such as accents are usually the same.\nThey can be either "composed" or "decomposed" into the component parts\nat the binary level.
\nThis composition and decomposition transform is useful for (at least) two reasons:
\nI use the unicode_normalization crate\nto do this "decompose and filter" operation.\nSpecifically, the UnicodeNormalization trait,\nwhich has helpers which will work on most string-like types.
You might notice there are four confusingly named methods in the trait:\nnfd, nfkd, nfc, and nfkc.\nThe nf stands for "normalization form".\nThese functions normalize your strings.\nc and d stand for composition and decomposition.\nThe composed form is, roughly, the more compact form,\nwhereas the decomposed form is the version where you separate the base from the modifiers,\nthe jamo from the syllables, etc.
We were already decomposing strings so that we could remove the diacritics, using form NFD.\nThis works great for diacritics and even Hangul,\nbut 𝐊𝐄𝐁𝐀𝐁 𝐊𝐈𝐍𝐆 𝐘𝐀𝐍𝐆𝐎𝐍 shows that we were missing something.
\nThat something is the k, which stands for "compatibility."\nYou can refer to Unicode Standard Annex #15\nfor a full definition,\nbut the intuition is that compatibility equivalence of two characters\nis a bit more permissive than the stricter canonical equivalence.\nBy reducing two characters (or strings) to their canonical form,\nyou will be able to tell if they represent the same "thing" with the same visual appearance,\nbehavior, semantic meaning, etc.\nCompatibility equivalence is a weaker form.
Compatibility equivalence is extremely useful in our quest for determining whether two nearby place names\nare a fuzzy match.\nIt reduces things like ligatures, superscripts, and width variations into a standard form.\nIn the case of "𝐊𝐄𝐁𝐀𝐁 𝐊𝐈𝐍𝐆 𝐘𝐀𝐍𝐆𝐎𝐍," compatibility decomposition transforms it into the ASCII\n"KEBAB KING YANGON."\nAnd now we can correctly coalesce the available information into a single search result.
\nHopefully this shines a light on one small corner of the complexities of unicode!
\n", "summary": "", "date_published": "2025-05-09T00:00:00-00:00", "image": "media/kebab-king-duplicates.png", "authors": [ { "name": "Ian Wagner", "url": "https://fosstodon.org/@ianthetechie", "avatar": "media/avi.jpeg" } ], "tags": [ "unicode", "rust" ], "language": "en" }, { "id": "https://ianwwagner.com//edns-client-subnet-and-geographic-dns.html", "url": "https://ianwwagner.com//edns-client-subnet-and-geographic-dns.html", "title": "EDNS Client-Subnet and Geographic DNS", "content_html": "DNS is a complex beast,\nso it's little surprise when I learn something new.\nToday I learned a bit more about the internals of how DNS works\nat various privacy-centric providers.
\nIt all started a few weeks ago when someone I follow on Mastodon\n(I'm very sorry I can't remember who)\nmentioned Quad9 as a privacy-friendly DNS solution.\nI think the context was that they were looking for alternatives\nto the US "big tech" vendors, like Cloudflare and Google.\nI eventually remembered it and switched by DNS a few weeks ago from 1.1.1.1,\nCloudflare's similar service.
\nFast forward to the present.\nI was frustrated that some page loads were REALLY slow.\nI couldn't figure out a clear pattern, but two sites which I visit a LOT\nare stadiamaps.com and docs.stadiamaps.com,\nsince it's kinda my job ;)\nThis had been going on far at least a week or two,\nbut I thought it was just something funky with my ISP,\nor maybe they were throttling me (I'm sure I'm a top 1% bandwidth user).
\nI had enough of the slowness this morning and was about to type up a thread in our internal Zulip chat\nasking our network guy to look into it on Monday.\nSo like any decent engineer, I popped up a web inspector and captured a HAR file\nso they could "see what I saw."
\nAnd what did I see?\nAfter a few minutes looking over it for obvious problems,\nI noticed that our marketing site was loading from our edge server in...\nJohannesburg?!\nAnd our docs site was coming from a server in Texas!\n(We include some HTTP headers which assist in debugging this sort of thing.)
\nWell, that's not right...\nI popped up dig in my terminal and verified that, indeed, the A records were resolving to servers\nlocated on the other side of the world.\nAnd then it hit me.\nI had changed my DNS settings recently!\nThat must have something to do with it!
We use AWS Route53 for our geographically aware DNS resolution needs.\nIt's the best product I've seen in the industry,\nso I assumed it wasn't their fault.\nThen I remembered something I read in the Quad9 FAQ\nabout EDNS Client-Subnet (also known as EDNS0 and ECS).\nThat seems relevant...
\nThe quick version is that some DNS resolvers can use a truncated version of your IP address\nto improve the quality of the results (giving a server near you).\nAmazon has a great detailed writeup.
\nThe trouble is that this info could theoretically be used (with some more details) to identify you,\nso many privacy-focused DNS solutions (including Quad9) disable this by default
\nQuad9 operates an alternate server with EDNS Client-Subnet enabled.\nI tried that and it gave the results I expected.\nBut this is not where the story ends!
\nIt turns out, Cloudflare, which I had been using previously,\nalso gave the "expected" results.\nBut they state very clearly in their FAQ\nthat they do not use EDNS Client-Subnet.\nWhat gives?
\nAt this point I'm speculating,\nbut I think that their network setup is a bit different.\nCloudflare is famous for having an extensive edge network,\nand I have a server very close by.\nMy guess is that they make all upstream queries to authoritative servers\nAND cache any results in the same nearby datacenter.\nThis would easily explain why they can still give a geographically relevant result,\nwithout sending your subnet to the authoritative server.
\nQuad9 on the other hand either doesn't have as many servers nearby (for fast routing),\nor perhaps they are sharing cache results globally.
\nAs I said though, this is all just speculation.\nIf anyone has more knowledge of how Cloudflare and Quad9 operate,\nlet me know and I'll update this post!
\n", "summary": "", "date_published": "2025-05-03T00:00:00-00:00", "image": "", "authors": [ { "name": "Ian Wagner", "url": "https://fosstodon.org/@ianthetechie", "avatar": "media/avi.jpeg" } ], "tags": [ "dns", "networking" ], "language": "en" } ] }