electric-snow.net

If you know me per­son­al­ly, you know that I love to rail on web tech­nolo­gies, es­pe­cial­ly JavaScript/EC­MAScript and its long list of WTFs. So why did I re­cent­ly tweet about a hy­brid ar­ti­cle/web app I wrote? Let me ex­plain. And spoil­er alert: I'm not plan­ning to switch to web dev any time soon.

So first off, what's the app all about? It is an ar­ti­cle with in­ter­ac­tive, 3D di­a­grams de­signed to help teach stu­dents who have had lit­tle to no con­tact with graph­ics pro­gram­ming about ho­mo­ge­neous co­or­di­nates and ra­tio­nal splines. If you haven't en­coun­tered ei­ther yet, please check it out first and tell me about your ex­pe­ri­ence be­fore read­ing on!

I took on this project as my in­di­vid­u­al teach­ing project in Mod­ule 3 of the High­er Ed­u­ca­tion Teach­ing Cer­tifi­cate of­fered by TU Darm­stadt. If you work at TU Darm­stadt and have any in­ter­est in teach­ing at all (or are su­per­vis­ing mas­ter's and bach­e­lor's the­ses), I high­ly rec­om­mend check­ing out the cour­ses they of­fer. By the way, if you are a PhD stu­dent there, many of the same cour­ses are of­fered by In­ge­ni­um as well. If you are at a dif­fer­ent uni­ver­si­ty, check out what they of­fer. Many have sim­i­lar pro­grams these days — at least in Ger­many.

The idea un­der­ly­ing the project is to help stu­dents gain an in­tu­itive — not just the­o­ret­i­cal — un­der­stand­ing of ho­mo­ge­neous co­or­di­nates and ra­tio­nal splines by sup­ple­ment­ing the the­o­ret­i­cal de­scrip­tion with in­ter­ac­tive, 3D di­a­grams. While oth­ers have cre­at­ed sim­i­lar ar­ti­cles with in­ter­ac­tive 2D di­a­grams, the ex­tra di­men­sion al­lowed me to take two-di­men­sion­al ho­mo­ge­neous co­or­di­nates, which are rep­re­sent­ed as three-di­men­sion­al vec­tors, and vi­su­al­ize the pro­jec­tion process as well. And while the fi­nal dis­play screen it­self is flat, hu­mans are pret­ty good at in­ter­pret­ing flat pro­jec­tions of 3D worlds, es­pe­cial­ly if the view­point can be changed. I guess you could al­so try do­ing some We­bXR shenani­gans for true stereo­scop­ic 3D vi­su­al­iza­tion, but that would have been way be­yond the scope of the project. Fur­ther­more, that would have gone against a sec­ondary goal I had in im­ple­ment­ing the project: as low a bar­ri­er of en­try as pos­si­ble.

Not ev­ery­one can af­ford their own com­put­er. Many on­ly have a smart­phone. Some may not even have that and on­ly use lab and li­brary com­put­ers on which they can't in­stall any­thing. While luck­i­ly in­creas­ing­ly rare in this coun­try, these stu­dents ex­ist even to­day, even in com­put­er sci­ence and re­lat­ed de­gree pro­grams, and we shouldn't ex­clude them — or any­one ex­cept those who would ex­clude oth­ers. Since an in­stal­lable app was out of the ques­tion, this re­al­ly left me with on­ly one op­tion: a Web ap­pli­ca­tion us­ing We­bGL (not We­bGL 2 or We­bG­PU, since that would run counter the goal of a low bar­ri­er of en­try). As ac­cess to tech­nol­o­gy isn't the on­ly fac­tor that may lim­it ac­ces­si­bil­i­ty, I tried my best to make the page as ac­ces­si­ble as pos­si­ble by:

• Us­ing se­man­tic HTM­L5
• Choos­ing col­ors that ful­fill con­trast guide­lines and re­spect the us­er's dark or light theme pref­er­ence
• Us­ing Ka­TeX for for­mu­la ren­der­ing, as it in­cludes MathML an­no­ta­tions, not on­ly an im­age or SVG like some oth­er sys­tems
• Mak­ing the points not on­ly ed­itable in 3D, but as tex­tu­al in­put fields as well
• Al­so ren­der­ing the pro­ject­ed po­si­tions as a ta­ble in the al­ter­nate fall­back con­tent of the can­vas el­e­ments

Is this suf­fi­cient? I hon­est­ly don't know, as I'm no ac­ces­si­bil­i­ty ex­pert, nor do I have any ex­pe­ri­ence us­ing screen read­ers or oth­er as­sis­tive tech­nolo­gies, but I tried my best — should you find an is­sue and ide­al­ly have a sug­ges­tion how to fix it, please file an is­sue.

Now, as I men­tioned up front: I am not nor do I plan to be­come a “Web de­vel­op­er,” so the de­vel­op­ment of the app was a bit of an odyssey of failed at­tempts, un­der­es­ti­mat­ed ef­forts, and prob­a­bly a good dose of un­usu­al (ques­tion­able?) ap­proach­es — the main ones prob­a­bly be­ing writ­ing raw HTM­L5 and CSS and not us­ing any “frame­works.” So let's have a look at the failed at­tempts be­fore we ad­dress the so­lu­tion I fi­nal­ly went with.

Failed At­tempt №1: Elm

The first thing I tried was Elm. Yes, be­lieve it or not, the first thing I went for was Elm, not Rust which I know nor JavaScript which al­most ev­ery­one us­es for these things. So why Elm? I like learn­ing new things! New pro­gram­ming lan­guages can of­ten teach you new ap­proach­es to solv­ing pro­gram­ming prob­lems. One such ap­proach — func­tion­al­ly re­ac­tive pro­gram­ming (FRP) — is a pure func­tion­al ap­proach to UI pro­gram­ming and pro­vides the foun­da­tion for Elm. Or so I thought.

It turns out they dropped FRP as a core part of Elm in 2016. And de­spite start­ing the project in 2020, most of the Elm tu­to­ri­als pop­ping up in my fil­ter bub­ble were still us­ing old­er, FRP-based Elm. This didn't stop me though, as Elm is still a pure, func­tion­al lan­guage with a syn­tax very close to Haskell, and I haven't done any func­tion­al pro­gram­ming in for­ev­er. The last time I wrote any­thing re­mote­ly close to pure func­tion­al code was a few lines of Rack­et (a LISP-de­riv­a­tive) in my first year of uni­ver­si­ty 16–17 years ago.

Since Elm has (ex­per­i­men­tal) We­bGL sup­port, I quick­ly had the ba­sics of draw­ing some­thing up and run­ning. I ran in­to some DPI-re­lat­ed mouse point­er han­dling is­sues that I had to work around by cre­at­ing my own JavaScript event han­dler that per­formed the nec­es­sary com­pu­ta­tions be­fore for­ward­ing the da­ta via an Elm port. But aside from these mi­nor tech­ni­cal is­sues, things seemed to go well, so I kept trudg­ing along, adding el­e­ment af­ter el­e­ment to my ren­der­er.

But af­ter a while, I no­ticed things were get­ting slow­er and slow­er, to the point of a sig­nif­i­cant drop in frame rate (think less than 30 FPS) on my desk­top ma­chine, which has a fair­ly beefy GPU, as I'm al­so a gamer. Not good if I want to keep it in­ter­ac­tive and keep the bar­ri­er of en­try low. While the Elm We­bGL li­brary ap­plies a few tricks to avoid re-up­load­ing mesh­es ev­ery frame, it failed in many cas­es. Why are these tricks even need­ed? Since Elm is a pure func­tion­al lan­guage, you don't mod­i­fy any state, but cre­ate a new state — in­clud­ing ev­ery­thing that should be ren­dered — ev­ery time. While Elm's lazy eval­u­a­tion helps to avoid a lot of com­pu­ta­tions, the struc­ture of the code and even the or­der of pa­ram­e­ters has a sig­nif­i­cant ef­fect on what is or isn't mem­o­ized. And what Elm We­bGL can mem­o­ize is even more lim­it­ed. I tried re­struc­tur­ing it a bit and was able to im­prove the frame rate slight­ly. How­ev­er, the nec­es­sary refac­tor­ing made the code more dif­fi­cult to un­der­stand not less. And even then, the frame rate was far from great, so I gave up on us­ing Elm.

That cer­tain­ly doesn't mean it was all bad, though. The Elm pro­gram­ming mod­el works re­al­ly well for more tra­di­tion­al ap­pli­ca­tions. And like Rust avoids mem­o­ry safe­ty bugs at com­pile time, Elm avoids run­time ex­cep­tions and catch­es al­most all er­rors that can hap­pen with reg­u­lar JavaScript at com­pile time. This even ex­tend­ed to the Elm We­bGL li­brary type check­ing shad­er ver­tex at­tributes and uni­forms at com­pile time! 🤯 Al­so like Rust, the Elm project puts a lot of ef­fort in­to pro­duc­ing help­ful and friend­ly er­ror mes­sages. At this point, a big shout-out and thank you to Es­te­ban Küber and ev­ery­one else work­ing on im­prov­ing Rust's er­ror mes­sages! ❤️

Failed At­tempt №2: Rust

For my sec­ond at­tempt, I de­cid­ed to go for some­thing more fa­mil­iar: Rust. Be­tween its strong cross-com­pi­la­tion sup­port, in­clud­ing three dif­fer­ent We­bAssem­bly (wasm32) tar­gets with Tier 2 sup­port, and the great work of the Rust and We­bAssem­bly do­main work­ing group, I thought it should be in­ter­est­ing to try us­ing Rust for graph­ics on the Web.

De­spite us­ing Rust, I wasn't able to avoid npm, webpack and oth­er parts of Web pro­gram­ming that I hate strong­ly dis­like with this ap­proach, but based off the web-sys We­bGL Ex­am­ple I was quick­ly able to be­gin draw­ing to the screen. How­ev­er, I then re­al­ized how much I would have to do and bind my­self that Elm We­bGL did for me, such as get­ting all the uni­form and ver­tex buf­fer types right. And event han­dling (for mouse events, for ex­am­ple) felt re­al­ly off with the fact that you have to box and for­get clo­sures… All solv­able is­sues that I have solved sev­er­al times for desk­top OpenGL al­ready, but this was a side project hap­pen­ing late in the evening af­ter 8+ hours of work or over the week­end, so I re­al­ly couldn't af­ford the amount of work it would en­tail.

I con­sid­ered us­ing an en­gine with We­bGL sup­port, but most en­gines aren't ex­act­ly de­signed for draw­ing 3D di­a­grams that are most­ly pro­gram­mat­i­cal­ly gen­er­at­ed and in­clude things like pre­scribed screen-space width lines. Fur­ther­more, I want­ed to have mul­ti­ple di­a­gram in­stances with dif­fer­ent in­put da­ta and most en­gines fo­cus on a sin­gle win­dow, and from what I could tell, mul­ti­ple in­stances of the same WASM mod­ule aren't sup­port­ed. Al­so, even in its min­i­mal form, the gen­er­at­ed WASM files were sur­pris­ing­ly large.

Since I was al­ready run­ning short on time, I had to cut my loss­es, stop faffing about, and use some­thing that just works. How­ev­er, I re­al­ly want to ex­plore Rust for in­ter­ac­tive graph­ics and the Web more at some point in the fu­ture (look­ing at you, wgpu-rs 👀)! For one of my non-in­ter­ac­tive (and non-Web) graph­ics ex­per­i­ments in Rust, check out l0calh05t/raytracing-weekend-rs.

The So­lu­tion: Type­Script with Three.js

The so­lu­tion I fi­nal­ly land­ed on was us­ing Type­Script with Three.js. While I don't usu­al­ly do Web de­vel­op­ment, some of my col­leagues use Three.js with Type­Script for 3D Web apps, so I knew it works well. Three.js ab­stracts away a lot of the de­tails, but al­lows pro­gram­mat­ic gen­er­a­tion of ge­om­e­try and sup­ports wide lines. Plus it sup­ports both We­bGL and We­bGL 2.

Com­pared to raw JavaScript, Type­Script has stat­ic typ­ing (which you can lo­cal­ly es­cape if nec­es­sary, which I had to do a few times where type dec­la­ra­tions were out of date or in­com­plete), which avoids a lot of JavaScript's in­san­i­ty — there­by al­low­ing you to keep your san­i­ty. Fur­ther­more, the tran­spiler can take ad­vanced/mod­ern ES6+ con­structs and com­pile them down to old­er, more wide­ly sup­port­ed ES stan­dards. So I found it a lot more com­fort­able to work with than raw JavaScript.

Most of the pain I had set­ting up the project, was that webpack v5 had fair­ly re­cent­ly sta­bi­lized, and like the Elm sit­u­a­tion be­fore, most in­for­ma­tion on the Web still re­ferred to webpack v4. But af­ter that, the rest went pret­ty smooth­ly. I even got a nice or­bit cam­era con­troller with full pinch-zoom sup­port, in­er­tia and ev­ery­thing else es­sen­tial­ly for free. In­ter­ac­tion with scene ob­jects was a bit of a has­sle though, since none of the li­braries I found worked out-of-the-box, ei­ther due to mis­match­es with the cur­rent (at the time) Three.js ver­sion or the lack of Type­Script type in­for­ma­tion. In the end, I had to use a lo­cal ven­dored copy of markuslerner/THREE.Interactive. But aside from these mi­nor is­sues, I can't com­plain!

That's all for now, and we'll re­turn to our reg­u­lar­ly sched­uled Rust pro­gram­ming next time. If you have been fol­low­ing along, you know the drill: feel free to fol­low me and send me a DM on Mastodon if you have any ques­tions or com­ments! I would be very hap­py to re­ceive feed­back re­gard­ing the app (es­pe­cial­ly should you de­cide to use it in a course!). While I have used it in a course and pre­pared a short feed­back ques­tion­naire, none of the stu­dents sub­mit­ted the ques­tion­naire, so I re­al­ly don't know if it was use­ful to any­one — or if it was pe­rused at all. At least the col­leagues to whom I showed it liked the con­cept and the de­sign.