<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Shane Anderson</title><link>https://shaneanderson.id.au/</link><description/><atom:link href="https://shaneanderson.id.au/feeds/all.rss.xml" rel="self"/><lastBuildDate>Wed, 22 Jul 2026 20:00:00 +1000</lastBuildDate><item><title>My Django layout refactor skill</title><link>https://shaneanderson.id.au/my-django-layout-refactor-skill.html</link><description>&lt;p&gt;I've created an agent Skill that automatically implements my preferred Django structure as I posted about &lt;a href="https://shaneanderson.id.au/how-i-structure-django-projects-for-better-maintainability-2025.html"&gt;previously&lt;/a&gt;, you can grab it &lt;a href="https://github.com/rxvt/skills/tree/main/django-restructure"&gt;here&lt;/a&gt; - saves me heaps of time when setting up a new Django project!&lt;/p&gt;</description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Shane Anderson</dc:creator><pubDate>Wed, 22 Jul 2026 20:00:00 +1000</pubDate><guid>tag:shaneanderson.id.au,2026-07-22:/my-django-layout-refactor-skill.html</guid><category>Projects</category><category>python</category><category>django</category><category>ai</category></item><item><title>Fixing Fronius solar inverter connectivity drop outs</title><link>https://shaneanderson.id.au/fixing-fronius-solar-inverter-connectivity-drop-outs.html</link><description>&lt;p&gt;From the moment my solar installation was completed I started having problems with my Fronius inverter dropping offline every few days, requiring a full hard restart to get it back on the network.&lt;/p&gt;
&lt;p&gt;I updated the firmware, etc with no improvement but then I noticed that the dropoff pattern was pretty consistent which got me wondering if it was having an issue renewing its DHCP lease. I updated the settings to use a static IP, reserved the IP on my router so nothing else would obtain it and the inverter has had a stable connection ever since.&lt;/p&gt;</description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Shane Anderson</dc:creator><pubDate>Sun, 12 Jul 2026 15:01:00 +1000</pubDate><guid>tag:shaneanderson.id.au,2026-07-12:/fixing-fronius-solar-inverter-connectivity-drop-outs.html</guid><category>General</category><category>misc</category></item><item><title>Use Super-C &amp; Super-V on Linux to copy &amp; paste like macOS</title><link>https://shaneanderson.id.au/use-super-c-super-v-on-linux-to-copy-paste-like-macos.html</link><description>&lt;p&gt;I like Linux, but one long-standing annoyance on the desktop has been that copy and paste in the terminal requires Ctrl-Shift-C and Ctrl-Shift-V as Ctrl-C is reserved for canceling jobs. Recently Omarchy &lt;a href="https://github.com/basecamp/omarchy/releases/tag/v3.1.0"&gt;implemented&lt;/a&gt; universal copy/paste using Super‑C and Super‑V, which fixes that annoyance, yay!&lt;/p&gt;
&lt;p&gt;For those that are unaware the Super key is the “Windows” or “Command” key on most keyboards. Combining Super‑C / Super‑V with swapping Super and Alt in Hyprland makes copy and paste behave the like it does on macOS which resolves a small, but annoying workflow issue for me and allowing me to keep the same muscle memory across MacOS and Linux.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;input{
  kb_options = altwin:swap_alt_win
}
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;</description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Shane Anderson</dc:creator><pubDate>Fri, 07 Nov 2025 09:59:00 +1100</pubDate><guid>tag:shaneanderson.id.au,2025-11-07:/use-super-c-super-v-on-linux-to-copy-paste-like-macos.html</guid><category>General</category><category>linux</category></item><item><title>Using the Unread RSS reader for YouTube</title><link>https://shaneanderson.id.au/using-the-unread-rss-reader-for-youtube.html</link><description>&lt;p&gt;I've been struggling with my YouTube addiction for a fair while now. I try to only use the Subscriptions view to limit myself to the latest videos from the channels I'm subscribed to but inevitably I get pulled down the algorithm rabbit hole and waste hours watching random videos. I've considered dropping YouTube altogether but there is so much great content on there that I don't want to miss out on, luckily I've found a better way.&lt;/p&gt;
&lt;p&gt;I've been using the &lt;a href="https://www.goldenhillsoftware.com/unread/"&gt;Unread&lt;/a&gt; RSS reader for a while now on my MacBook and iPhone and it's a great piece of software, but I recently discovered you can also use it to subscribe to YouTube channels in addition to RSS feeds! After adding my YouTube subscriptions it's definitely become my favourite way to consume YouTube content, and by default I only see a short text description of the video and there's no algorithm trying to cram more videos down my throat - it's great!&lt;/p&gt;
&lt;p&gt;&lt;img alt="Screenshot" src="https://shaneanderson.id.au/images/using-unread-rss-reader-for-youtube/unread_rss_youtube.jpeg"&gt;&lt;/p&gt;</description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Shane Anderson</dc:creator><pubDate>Sat, 13 Sep 2025 15:29:00 +1000</pubDate><guid>tag:shaneanderson.id.au,2025-09-13:/using-the-unread-rss-reader-for-youtube.html</guid><category>General</category><category>youtube</category><category>rss</category></item><item><title>EXAPUNKS cheat sheet</title><link>https://shaneanderson.id.au/exapunks-cheat-sheet.html</link><description>&lt;p&gt;&lt;a href="https://store.steampowered.com/app/716490/EXAPUNKS/"&gt;EXAPUNKS&lt;/a&gt; is one of my favourite games, and it's nice to have a &lt;a href="/static/exapunks_cheatsheet.html"&gt;cheat sheet&lt;/a&gt; up on my second monitor while playing so I can remember the keywords, etc so I created one.&lt;/p&gt;</description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Shane Anderson</dc:creator><pubDate>Sat, 23 Aug 2025 12:21:00 +1000</pubDate><guid>tag:shaneanderson.id.au,2025-08-23:/exapunks-cheat-sheet.html</guid><category>General</category><category>gaming</category></item><item><title>How I structure Django projects for better maintainability 2025</title><link>https://shaneanderson.id.au/how-i-structure-django-projects-for-better-maintainability-2025.html</link><description>&lt;p&gt;When starting a new Django project I've found myself tweaking the default project structure to make it more maintainable and organized. Here's my preferred layout and the small adjustments needed to make it work from an empty directory through to a running project.&lt;/p&gt;
&lt;p&gt;The default Django layout works fine, but I prefer using a &lt;code&gt;src&lt;/code&gt; directory to cleanly separate the Django application code from project configuration files (&lt;code&gt;pyproject.toml&lt;/code&gt;, &lt;code&gt;README.md&lt;/code&gt;, etc.).&lt;/p&gt;
&lt;h2&gt;Why I Prefer This Structure&lt;/h2&gt;
&lt;p&gt;I make a few organizational changes that help keep larger projects maintainable:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Rename the project directory to &lt;code&gt;config&lt;/code&gt;&lt;/strong&gt; - This makes it immediately clear that this directory contains configuration rather than application logic.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Split settings and URLs into subdirectories&lt;/strong&gt; - &lt;code&gt;config/settings/&lt;/code&gt; and &lt;code&gt;config/urls/&lt;/code&gt; allow for clean environment-specific configurations (local, staging, production).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Create a dedicated &lt;code&gt;apps&lt;/code&gt; directory&lt;/strong&gt; - This keeps all your Django applications organized in one place, separate from configuration.&lt;/li&gt;
&lt;/ol&gt;
&lt;h2&gt;Setting Up the Project&lt;/h2&gt;
&lt;p&gt;These steps assume you have Django installed (for example, via &lt;code&gt;uv add django&lt;/code&gt; or &lt;code&gt;pip install django&lt;/code&gt;).&lt;/p&gt;
&lt;h3&gt;1. Create the layout&lt;/h3&gt;
&lt;p&gt;Create the repository directory and the &lt;code&gt;src/&amp;lt;projectname&amp;gt;&lt;/code&gt; directory that will hold your Django project:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;mkdir&lt;span class="w"&gt; &lt;/span&gt;myproject
&lt;span class="nb"&gt;cd&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;myproject
mkdir&lt;span class="w"&gt; &lt;/span&gt;-p&lt;span class="w"&gt; &lt;/span&gt;src/myproject
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;The outer directory is your repository or project name. The inner &lt;code&gt;config&lt;/code&gt; package is what &lt;code&gt;django-admin startproject&lt;/code&gt; would normally have named after your project; we rename it to &lt;code&gt;config&lt;/code&gt; to make its purpose obvious.&lt;/p&gt;
&lt;h3&gt;2. Run &lt;code&gt;django-admin startproject&lt;/code&gt;&lt;/h3&gt;
&lt;p&gt;Run the startproject command with &lt;code&gt;config&lt;/code&gt; as the project name:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;django-admin&lt;span class="w"&gt; &lt;/span&gt;startproject&lt;span class="w"&gt; &lt;/span&gt;config&lt;span class="w"&gt; &lt;/span&gt;src/myproject
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;This creates:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;src/myproject/manage.py&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;src/myproject/config/__init__.py&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;src/myproject/config/settings.py&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;src/myproject/config/urls.py&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;src/myproject/config/asgi.py&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;src/myproject/config/wsgi.py&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;3. Create the &lt;code&gt;apps&lt;/code&gt; directory&lt;/h3&gt;
&lt;p&gt;Create a directory for your applications:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;mkdir&lt;span class="w"&gt; &lt;/span&gt;-p&lt;span class="w"&gt; &lt;/span&gt;src/myproject/apps
touch&lt;span class="w"&gt; &lt;/span&gt;src/myproject/apps/__init__.py
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3&gt;4. Convert &lt;code&gt;config/settings.py&lt;/code&gt; into a package&lt;/h3&gt;
&lt;p&gt;Create a &lt;code&gt;settings&lt;/code&gt; package and move the generated file into it:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;mkdir&lt;span class="w"&gt; &lt;/span&gt;src/myproject/config/settings
touch&lt;span class="w"&gt; &lt;/span&gt;src/myproject/config/settings/__init__.py
mv&lt;span class="w"&gt; &lt;/span&gt;src/myproject/config/settings.py&lt;span class="w"&gt; &lt;/span&gt;src/myproject/config/settings/default.py
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Create a local settings file:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# src/myproject/config/settings/local.py&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nn"&gt;.default&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3&gt;5. Convert &lt;code&gt;config/urls.py&lt;/code&gt; into a package&lt;/h3&gt;
&lt;p&gt;Do the same for URLs:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;mkdir&lt;span class="w"&gt; &lt;/span&gt;src/myproject/config/urls
touch&lt;span class="w"&gt; &lt;/span&gt;src/myproject/config/urls/__init__.py
mv&lt;span class="w"&gt; &lt;/span&gt;src/myproject/config/urls.py&lt;span class="w"&gt; &lt;/span&gt;src/myproject/config/urls/default.py
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;You can optionally create a local URL file:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# src/myproject/config/urls/local.py&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nn"&gt;.default&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;urlpatterns&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3&gt;6. Update generated files that reference the old settings module&lt;/h3&gt;
&lt;p&gt;After renaming the project package to &lt;code&gt;config&lt;/code&gt;, the settings module is no longer &lt;code&gt;myproject.settings&lt;/code&gt; -- it is &lt;code&gt;config.settings.local&lt;/code&gt;. The following files were generated by &lt;code&gt;django-admin startproject&lt;/code&gt; and still contain the original default:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;src/myproject/manage.py&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;src/myproject/config/wsgi.py&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;src/myproject/config/asgi.py&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In each file, change:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;setdefault&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;DJANGO_SETTINGS_MODULE&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;myproject.settings&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;to:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;setdefault&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;DJANGO_SETTINGS_MODULE&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;&amp;#39;config.settings.local&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Tip:&lt;/strong&gt; Setting this directly in the generated files means &lt;code&gt;manage.py check&lt;/code&gt; and &lt;code&gt;runserver&lt;/code&gt; work out of the box without relying on an environment variable or &lt;code&gt;.envrc&lt;/code&gt;. The &lt;code&gt;DJANGO_SETTINGS_MODULE&lt;/code&gt; environment variable can still be used to override it (for example, in production or staging).&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h3&gt;7. Set the &lt;code&gt;DJANGO_SETTINGS_MODULE&lt;/code&gt; environment variable&lt;/h3&gt;
&lt;p&gt;If you use &lt;code&gt;direnv&lt;/code&gt;, add this to &lt;code&gt;.envrc&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# .envrc&lt;/span&gt;
&lt;span class="nb"&gt;export&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;DJANGO_SETTINGS_MODULE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;config.settings.local&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Or set it manually whenever you work on the project:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nb"&gt;export&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;DJANGO_SETTINGS_MODULE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;config.settings.local&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h2&gt;Required Configuration Changes&lt;/h2&gt;
&lt;p&gt;A few small changes inside &lt;code&gt;config/settings/default.py&lt;/code&gt; are still required.&lt;/p&gt;
&lt;h3&gt;1. Adjust &lt;code&gt;BASE_DIR&lt;/code&gt;&lt;/h3&gt;
&lt;p&gt;Because the &lt;code&gt;config/settings&lt;/code&gt; directory is now nested one level deeper, &lt;code&gt;BASE_DIR&lt;/code&gt; needs an additional &lt;code&gt;.parent&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# config/settings/default.py&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nn"&gt;pathlib&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Path&lt;/span&gt;

&lt;span class="n"&gt;BASE_DIR&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Path&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="vm"&gt;__file__&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;resolve&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parent&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parent&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parent&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3&gt;2. Update &lt;code&gt;ROOT_URLCONF&lt;/code&gt;&lt;/h3&gt;
&lt;p&gt;Since &lt;code&gt;urls.py&lt;/code&gt; was moved into a package, Django needs the new module path:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# config/settings/default.py&lt;/span&gt;
&lt;span class="n"&gt;ROOT_URLCONF&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;&amp;quot;config.urls.default&amp;quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3&gt;3. Add the apps directory to the Python path&lt;/h3&gt;
&lt;p&gt;Also in &lt;code&gt;config/settings/default.py&lt;/code&gt;, add the &lt;code&gt;apps&lt;/code&gt; directory to the Python path so Django can find your applications with short names:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="c1"&gt;# config/settings/default.py&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nn"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nn"&gt;sys&lt;/span&gt;

&lt;span class="n"&gt;sys&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;insert&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;BASE_DIR&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;&amp;quot;apps&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; Mutating &lt;code&gt;sys.path&lt;/code&gt; is convenient for short app names like &lt;code&gt;myapp&lt;/code&gt;, but it is order-sensitive and can cause confusing import errors. An alternative is to avoid &lt;code&gt;sys.path&lt;/code&gt; changes and import apps using their full package path (&lt;code&gt;myproject.apps.myapp&lt;/code&gt;).&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2&gt;Running the Project&lt;/h2&gt;
&lt;p&gt;Because &lt;code&gt;manage.py&lt;/code&gt; is nested inside &lt;code&gt;src/myproject/&lt;/code&gt;, run commands like this:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nb"&gt;cd&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;src/myproject
python&lt;span class="w"&gt; &lt;/span&gt;./manage.py&lt;span class="w"&gt; &lt;/span&gt;check
python&lt;span class="w"&gt; &lt;/span&gt;./manage.py&lt;span class="w"&gt; &lt;/span&gt;migrate
python&lt;span class="w"&gt; &lt;/span&gt;./manage.py&lt;span class="w"&gt; &lt;/span&gt;runserver
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h2&gt;Example Directory Structure&lt;/h2&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;.
├── Justfile
├── README.md
├── pyproject.toml
├── src
│   └── myproject
│       ├── __init__.py
│       ├── apps
│       │   ├── __init__.py
│       │   └── mycoolapp
│       │       ├── __init__.py
│       │       ├── admin.py
│       │       ├── apps.py
│       │       ├── migrations
│       │       │   └── __init__.py
│       │       ├── models.py
│       │       ├── templates
│       │       │   └── mycoolapp
│       │       │       └── ...
│       │       ├── tests.py
│       │       ├── urls.py
│       │       └── views.py
│       ├── config
│       │   ├── __init__.py
│       │   ├── asgi.py
│       │   ├── settings
│       │   │   ├── __init__.py
│       │   │   ├── default.py
│       │   │   └── local.py
│       │   ├── urls
│       │   │   ├── __init__.py
│       │   │   ├── default.py
│       │   │   └── local.py
│       │   └── wsgi.py
│       ├── db.sqlite3
│       └── manage.py
└── uv.lock
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h2&gt;Verification&lt;/h2&gt;
&lt;p&gt;Run the Django system check:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="nb"&gt;cd&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;src/myproject
python&lt;span class="w"&gt; &lt;/span&gt;./manage.py&lt;span class="w"&gt; &lt;/span&gt;check
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Expected output:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;System check identified no issues (0 silenced).
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;</description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Shane Anderson</dc:creator><pubDate>Sun, 17 Aug 2025 19:07:00 +1000</pubDate><guid>tag:shaneanderson.id.au,2025-08-17:/how-i-structure-django-projects-for-better-maintainability-2025.html</guid><category>General</category><category>python</category><category>django</category></item><item><title>Making Change Text in Vim a bit more sane</title><link>https://shaneanderson.id.au/making-change-text-in-vim-a-bit-more-sane.html</link><description>&lt;p&gt;I've been finding it super annoying that Change Text in Vim yanks the changed text to the default register, overwriting the text I had stored. I 
wanted something more akin to how copy and paste works on Windows or MacOS where I can have some text stored in the buffer and write on top of the
existing text - so I asked &lt;a href="https://anthropic.com/"&gt;Claude.AI&lt;/a&gt; and it gave me the following which works perfectly.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="kr"&gt;return&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;LazyVim&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nv"&gt;opts&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kr"&gt;function&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="c1"&gt;-- Custom mappings for change command without affecting registers&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="nv"&gt;vim&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="py"&gt;keymap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;n&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;c&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;&amp;quot;_c&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;noremap&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;desc&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Change without yanking&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="nv"&gt;vim&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="py"&gt;keymap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;n&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;C&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;&amp;quot;_C&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;noremap&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;desc&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Change to end of line without yanking&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="nv"&gt;vim&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="py"&gt;keymap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;n&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;cc&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;&amp;quot;_cc&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;noremap&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;desc&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Change entire line without yanking&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="nv"&gt;vim&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="py"&gt;keymap&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;x&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;c&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;&amp;#39;&amp;quot;_c&amp;#39;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;noremap&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;desc&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Change selection without yanking&amp;quot;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;

&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="c1"&gt;-- Keep normal delete/cut behavior unchanged&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="c1"&gt;-- d, dd, D will still yank to registers as normal&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="kr"&gt;end&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;</description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Shane Anderson</dc:creator><pubDate>Sun, 10 Aug 2025 09:05:00 +1000</pubDate><guid>tag:shaneanderson.id.au,2025-08-10:/making-change-text-in-vim-a-bit-more-sane.html</guid><category>TIL</category><category>vim</category></item><item><title>Adding Kagi as the default search engine in Alfred</title><link>https://shaneanderson.id.au/adding-kagi-as-the-default-search-engine-in-alfred.html</link><description>&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Add Kagi as a custom search following the guide &lt;a href="https://help.kagi.com/kagi/getting-started/application-integrations.html"&gt;here&lt;/a&gt;.
 &lt;img alt="Adding Kagi as a custom search in Alfred" src="https://shaneanderson.id.au/images/adding-kagi-as-default-search-engine-in-alfred/kaji_alfred_1.webp"&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;In the Default Results section click on the &lt;code&gt;Setup fallback results&lt;/code&gt; button.
&lt;img alt="Alfred default results with setup fallback results button" src="https://shaneanderson.id.au/images/adding-kagi-as-default-search-engine-in-alfred/kaji_alfred_2.webp"&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Click on the &lt;code&gt;+&lt;/code&gt; button, then &lt;code&gt;Custom Search&lt;/code&gt; and select Kagi from the list. Drag it to the first position if you want Kagi to be your default.
&lt;img alt="Adding Kagi as a fallback search option" src="https://shaneanderson.id.au/images/adding-kagi-as-default-search-engine-in-alfred/kaji_alfred_3.webp"&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;</description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Shane Anderson</dc:creator><pubDate>Sat, 09 Aug 2025 12:00:00 +1000</pubDate><guid>tag:shaneanderson.id.au,2025-08-09:/adding-kagi-as-the-default-search-engine-in-alfred.html</guid><category>TIL</category><category>macos</category></item><item><title>How to create snippets in LazyVim</title><link>https://shaneanderson.id.au/how-to-create-snippets-in-lazyvim.html</link><description>&lt;p&gt;Took me way too much effort to figure this out and searching for info resulted in a lot of posts with  other people asking the same question or outdated information as LazyVim has changed its plugins a bunch of times, hopefully this helps someone else out there.&lt;/p&gt;
&lt;p&gt;This example will use the template I'm using for my blog posts and is compatible with the &lt;a href="https://github.com/saghen/blink.cmp"&gt;Blink&lt;/a&gt; plugin that LazyVim is using for completions (under &lt;a href="https://www.lazyvim.org/extras/coding/blink"&gt;Extras&lt;/a&gt;) in 2025.&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;mkdir&lt;span class="w"&gt; &lt;/span&gt;~/.config/nvim/snippets
touch&lt;span class="w"&gt; &lt;/span&gt;~/.config/nvim/snippets/markdown.json
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;pelican-post&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;prefix&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;post&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;body&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Title: ${1:Post Title}&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Date: $CURRENT_YEAR-$CURRENT_MONTH-$CURRENT_DATE $CURRENT_HOUR:$CURRENT_MINUTE&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Category: ${2:General, Projects}&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Tags: ${3:aws, python, vim}&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Status: ${4:draft, published, hidden, skip}&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Summary: ${5:Brief description}&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="w"&gt;      &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;$0&amp;quot;&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
&lt;span class="w"&gt;    &lt;/span&gt;&lt;span class="nt"&gt;&amp;quot;description&amp;quot;&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;quot;Pelican blog post&amp;quot;&lt;/span&gt;
&lt;span class="w"&gt;  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;Now you should be able to open up a Markdown file, type &lt;code&gt;post&amp;lt;Tab&amp;gt;&lt;/code&gt; and have the snippet appear. You should be able to tab through the fields too.&lt;/p&gt;</description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Shane Anderson</dc:creator><pubDate>Sat, 02 Aug 2025 18:53:00 +1000</pubDate><guid>tag:shaneanderson.id.au,2025-08-02:/how-to-create-snippets-in-lazyvim.html</guid><category>TIL</category><category>vim</category></item><item><title>Introducing S3Fetch</title><link>https://shaneanderson.id.au/introducing-s3fetch.html</link><description>&lt;h2&gt;Why S3Fetch?&lt;/h2&gt;
&lt;p&gt;If you've ever tried to download a large number of objects from S3 using the AWS CLI, you've probably noticed it can be painfully slow. The AWS CLI's &lt;code&gt;aws s3 sync&lt;/code&gt; or &lt;code&gt;aws s3 cp --recursive&lt;/code&gt; commands process objects sequentially and don't start downloading until the complete object listing is finished.&lt;/p&gt;
&lt;p&gt;S3Fetch solves this by:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Concurrent operations&lt;/strong&gt;: Lists objects and downloads files simultaneously in separate threads&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Immediate downloads&lt;/strong&gt;: Starts downloading the first object as soon as it's discovered, while still listing remaining objects&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Optimized for prefixes&lt;/strong&gt;: Only requests objects under your specified prefix, reducing API calls&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Configurable threading&lt;/strong&gt;: Tune performance with adjustable thread counts&lt;/li&gt;
&lt;/ul&gt;
&lt;div class="admonition note"&gt;
&lt;p class="admonition-title"&gt;Note&lt;/p&gt;
&lt;p&gt;I wrote the first version of S3Fetch when I wasn't a very good programmer,
and arguably I'm still not, but I have been working on a version 2 which gets
rid of the most egregious mistakes such as the God Class, etc. That code has
been pushed to &lt;code&gt;main&lt;/code&gt; on the &lt;a href="https://github.com/rxvt/s3fetch"&gt;GitHub repo&lt;/a&gt; and
it works, but I haven't gotten around to releasing a new package yet.&lt;/p&gt;
&lt;/div&gt;
&lt;h2&gt;How It Works&lt;/h2&gt;
&lt;p&gt;The basic idea is pretty simple: why wait around?&lt;/p&gt;
&lt;p&gt;Most tools (like AWS CLI) do this:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Ask S3 for a list of ALL your files&lt;/li&gt;
&lt;li&gt;Wait... and wait... for the complete list&lt;/li&gt;
&lt;li&gt;Finally start downloading files one by one&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;S3Fetch does this instead:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Start asking S3 for your file list in one thread&lt;/li&gt;
&lt;li&gt;As soon as S3 mentions the first file, start downloading it in another thread&lt;/li&gt;
&lt;li&gt;Keep downloading files in multiple threads while the first thread is still
getting the rest of the list&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;So if you're downloading 1,000 files, S3Fetch is already downloading file #1
(and #2, #3, etc.) in separate threads while the main thread is still figuring
out what files #500-1000 even are.&lt;/p&gt;
&lt;p&gt;That's basically it - just use separate threads so you don't wait around when
you don't have to.&lt;/p&gt;
&lt;h3&gt;Performance Comparison&lt;/h3&gt;
&lt;p&gt;For downloading 428 objects from a bucket with 12+ million objects:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;AWS CLI&lt;/strong&gt;: ~2-3 minutes (sequential listing + downloading)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;S3Fetch (8 threads)&lt;/strong&gt;: 29 seconds&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;S3Fetch (100 threads)&lt;/strong&gt;: 8 seconds&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;S3Fetch is Open Source, written in Python and you can read more detailed
instructions and information, raise issues or contribute if that takes your
fancy via the &lt;a href="https://github.com/rxvt/s3fetch"&gt;GitHub repo&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;You can also use the comment section below if you just want to chat about S3Fetch
or ask questions.&lt;/p&gt;
&lt;h2&gt;Installation&lt;/h2&gt;
&lt;p&gt;Quick and easy install from PyPi. I recommend installation using &lt;a href="https://pypi.org/project/pipx/"&gt;pipx&lt;/a&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;pipx install s3fetch
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;You can also install using &lt;code&gt;pip&lt;/code&gt;:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;pip install s3fetch
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h2&gt;Usage&lt;/h2&gt;
&lt;p&gt;Basic usage is simple:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;s3fetch s3://bucket/prefix
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3&gt;Common Examples&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;Download logs from a specific date:&lt;/strong&gt;&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;s3fetch s3://my-logs-bucket/application-logs/2021-01-28
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Download all files in a subdirectory:&lt;/strong&gt;&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;s3fetch s3://my-data-bucket/user-uploads/images/
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Download with custom thread count for better performance:&lt;/strong&gt;&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;s3fetch s3://large-bucket/big-dataset/ --threads 50
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Download to a specific local directory:&lt;/strong&gt;&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;s3fetch s3://my-bucket/data/ --output-dir ./downloads/
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3&gt;When to Use S3Fetch&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Large file counts&lt;/strong&gt;: Downloading hundreds or thousands of files&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Deep prefixes&lt;/strong&gt;: When your files are buried deep in a bucket with millions
  of objects&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Bandwidth optimization&lt;/strong&gt;: Making full use of your connection with parallel
  downloads&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;Command Options&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;--threads N&lt;/code&gt;: Number of download threads (default: CPU cores)&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--output-dir PATH&lt;/code&gt;: Download to specific directory (default: current
  directory)&lt;/li&gt;
&lt;li&gt;&lt;code&gt;--dry-run&lt;/code&gt;: Show what would be downloaded without actually downloading&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;You can find more detailed instructions in the
&lt;a href="https://github.com/rxvt/s3fetch"&gt;README&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;Benchmarks&lt;/h2&gt;
&lt;p&gt;Downloading 428 objects under the &lt;code&gt;fake-prod-data/2020-10-17&lt;/code&gt; prefix from a
bucket containing a total of 12,204,097 objects.&lt;/p&gt;
&lt;h3&gt;With 100 threads&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;s3fetch s3://fake-test-bucket/fake-prod-data/2020-10-17  --threads 100

8.259 seconds
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;

&lt;h3&gt;With 8 threads&lt;/h3&gt;
&lt;div class="highlight"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;code&gt;s3fetch s3://fake-test-bucket/fake-prod-data/2020-10-17  --threads 8

29.140 seconds
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;</description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Shane Anderson</dc:creator><pubDate>Thu, 28 Jan 2021 19:00:00 +1100</pubDate><guid>tag:shaneanderson.id.au,2021-01-28:/introducing-s3fetch.html</guid><category>Projects</category><category>aws</category><category>python</category></item></channel></rss>