<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>fz42</title><link>https://fz42.net/</link><description>Recent content on fz42</description><generator>Hugo -- gohugo.io</generator><language>en</language><lastBuildDate>Thu, 04 Sep 2025 00:00:00 +0000</lastBuildDate><atom:link href="https://fz42.net/feed.xml" rel="self" type="application/rss+xml"/><item><title>Introducing AI Influence Levels on fz42.net</title><link>https://fz42.net/posts/introducing-ai-influence-levels/</link><pubDate>Thu, 04 Sep 2025 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/introducing-ai-influence-levels/</guid><description>&lt;p&gt;Transparency matters. As AI becomes increasingly integrated into content creation, readers deserve to know the origin and nature of what they&amp;rsquo;re consuming. Today, I&amp;rsquo;m introducing &lt;strong&gt;AI Influence Levels (AIL)&lt;/strong&gt; to fz42.net, which is &lt;a href="https://danielmiessler.com/blog/ai-influence-level-ail"&gt;Daniel Miessler&amp;rsquo;s framework&lt;/a&gt; for rating content based on AI involvement.&lt;/p&gt;
&lt;h2 id="what-are-ai-influence-levels"&gt;What Are AI Influence Levels?&lt;/h2&gt;
&lt;p&gt;AI Influence Levels provide a standardized way to communicate how much AI was involved in creating content. Again, reminder that &lt;strong&gt;Daniel Miessler&lt;/strong&gt; created this framework, and I&amp;rsquo;m just using it here. The system uses a simple 0-5 scale:&lt;/p&gt;</description><content:encoded><![CDATA[<p>Transparency matters. As AI becomes increasingly integrated into content creation, readers deserve to know the origin and nature of what they&rsquo;re consuming. Today, I&rsquo;m introducing <strong>AI Influence Levels (AIL)</strong> to fz42.net, which is <a href="https://danielmiessler.com/blog/ai-influence-level-ail">Daniel Miessler&rsquo;s framework</a> for rating content based on AI involvement.</p>
<h2 id="what-are-ai-influence-levels">What Are AI Influence Levels?</h2>
<p>AI Influence Levels provide a standardized way to communicate how much AI was involved in creating content. Again, reminder that <strong>Daniel Miessler</strong> created this framework, and I&rsquo;m just using it here. The system uses a simple 0-5 scale:</p>
<ul>
<li><strong>AIL 0</strong>: Human Created, No AI Involved</li>
<li><strong>AIL 1</strong>: Human Created, Minor AI Assistance</li>
<li><strong>AIL 2</strong>: Human Created, Major AI Augmentation</li>
<li><strong>AIL 3</strong>: AI Created, Human Full Structure</li>
<li><strong>AIL 4</strong>: AI Created, Human Basic Idea</li>
<li><strong>AIL 5</strong>: AI Created, Little Human Involvement</li>
</ul>
<h2 id="why-this-matters">Why This Matters</h2>
<p>In an era where AI can generate everything from blog posts to artwork, the line between human and machine creativity is increasingly blurred. Readers care about authenticity and origin, especially when that origin might not be entirely human.</p>
<p>This isn&rsquo;t about judging whether AI assistance is &ldquo;good&rdquo; or &ldquo;bad.&rdquo; It&rsquo;s about transparency. Some readers prefer purely human-written content, while others are curious about AI-augmented creativity. By clearly labeling content with its AIL rating, everyone can make informed decisions about what they&rsquo;re reading.</p>
<h2 id="how-it-works-on-fz42net">How It Works on fz42.net</h2>
<p>Starting today, you&rsquo;ll notice a small ⚡ icon in the metadata section below post titles, followed by an AIL rating (e.g., &ldquo;AIL 1&rdquo;). Hover over the indicator to see a tooltip explaining what that level means.</p>
<p><img loading="lazy" src="/posts/introducing-ai-influence-levels/images/ail-level.png" type="" alt=""  /></p>
<p>Every post on the site will display an AIL rating:</p>
<ul>
<li>Posts without explicit ratings default to <strong>AIL 0</strong> (human-only)</li>
<li>Posts with AI involvement will show their appropriate level</li>
<li>The rating reflects the content creation process, not the ideas or expertise behind it</li>
</ul>
<h2 id="my-commitment-to-transparency">My Commitment to Transparency</h2>
<p>Most content on fz42.net will likely fall into AIL 0-2 range. I believe in human expertise and perspective, but I&rsquo;m not opposed to using AI tools for editing, research assistance, or exploring ideas. When I do use AI in any meaningful way, you&rsquo;ll know about it.</p>
<p>This post itself is rated <strong>AIL 2</strong>—I wrote the initial draft and structure, but used AI assistance to refine the language and ensure clarity. The ideas, perspective, and technical implementation are entirely my own.</p>
<h2 id="technical-implementation">Technical Implementation</h2>
<p>For those curious about the technical side, the AIL system is implemented as:</p>
<ol>
<li>An optional <code>ai_influence_level</code> parameter in post front matter</li>
<li>Hugo template logic that displays the appropriate indicator</li>
<li>Consistent visual design using the ⚡ icon across all levels</li>
<li>Descriptive tooltips explaining each level</li>
</ol>
<p>The feature is backward-compatible—existing posts without AIL ratings automatically display as AIL 0.</p>
<h2 id="looking-forward">Looking Forward</h2>
<p>As AI tools evolve and become more sophisticated, having clear standards for disclosure becomes increasingly important. By adopting Daniel Miessler&rsquo;s AIL framework, fz42.net joins a growing movement toward transparency in AI-assisted content creation.</p>
<p>I encourage other content creators to consider implementing similar disclosure systems. Transparency builds trust, and trust is the foundation of any meaningful reader relationship.</p>
]]></content:encoded></item><item><title>Automating My GitHub Profile with Actions</title><link>https://fz42.net/posts/automating-my-github-profile-with-actions/</link><pubDate>Mon, 28 Jul 2025 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/automating-my-github-profile-with-actions/</guid><description>&lt;p&gt;Two years ago I bookmarked &lt;a href="https://brandur.org/fragments/self-updating-github-readme"&gt;Building a self-updating GitHub README&lt;/a&gt;. Finally crossed it off my todo list this week.&lt;/p&gt;
&lt;h2 id="the-journey"&gt;The Journey&lt;/h2&gt;
&lt;p&gt;Four months ago, my employer got acquired. Anyone who&amp;rsquo;s been through an acquisition knows the drill - uncertainty about the future naturally leads to resume polishing.&lt;/p&gt;
&lt;p&gt;I got tired of installing TeX distributions just to build my resume. I created a GitHub Action that built the PDF as an artifact. Good enough&amp;hellip; I was manually downloading it in case I had to start sending it around.&lt;/p&gt;</description><content:encoded><![CDATA[<p>Two years ago I bookmarked <a href="https://brandur.org/fragments/self-updating-github-readme">Building a self-updating GitHub README</a>. Finally crossed it off my todo list this week.</p>
<h2 id="the-journey">The Journey</h2>
<p>Four months ago, my employer got acquired. Anyone who&rsquo;s been through an acquisition knows the drill - uncertainty about the future naturally leads to resume polishing.</p>
<p>I got tired of installing TeX distributions just to build my resume. I created a GitHub Action that built the PDF as an artifact. Good enough&hellip; I was manually downloading it in case I had to start sending it around.</p>
<p>This week, I decided to tackle the profile automation. I realized those artifacts expire after 90 days (by default) and I wouldn&rsquo;t be able to link to expiring artifacts. Time to evolve the system.</p>
<h2 id="the-private-repo-problem">The Private Repo Problem</h2>
<p>I upgraded to creating proper releases. But my resume repo is private. The automation worked great&hellip; for me. I was authenticated, so the PDF links worked. Everyone else got 404s.</p>
<p>Could I just make the resume repo public? Sure. But it needs cleanup first, and more importantly, I wanted to learn how to handle this when making a repo public isn&rsquo;t an option.</p>
<p>I discovered <code>repository_dispatch</code> which lets one GitHub repo trigger workflows in another using a Personal Access Token.</p>
<h2 id="why-the-hard-way">Why the Hard Way?</h2>
<p>This cross-repo integration pattern is valuable beyond my use case. What about:</p>
<ul>
<li>Client codebases that must stay private</li>
<li>Internal tools with public documentation</li>
<li>Any other scenario where &ldquo;just make it public&rdquo; isn&rsquo;t possible</li>
</ul>
<p>Learning <code>repository_dispatch</code> now means I have this tool ready when I actually need it.</p>
<h2 id="how-it-works">How It Works</h2>
<p><strong>Resume repo</strong> (private):</p>
<ol>
<li>Tag a release: <code>git tag v1.0.3</code></li>
<li>Action builds PDF and creates release</li>
<li>Triggers profile repo update</li>
</ol>
<p><strong>Profile repo</strong> (public):</p>
<ol>
<li>Receives dispatch event</li>
<li>Downloads PDF from private release</li>
<li>Commits PDF locally</li>
<li>Updates README</li>
</ol>
<h2 id="the-technical-details">The Technical Details</h2>
<p>In the resume workflow, after creating the release:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl">- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Trigger Profile Repository Update</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">if</span><span class="p">:</span><span class="w"> </span><span class="l">startsWith(github.ref, &#39;refs/tags/v&#39;)</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">      curl -X POST \
</span></span></span><span class="line"><span class="cl"><span class="sd">        -H &#34;Authorization: token ${{ secrets.PAT_TOKEN }}&#34; \
</span></span></span><span class="line"><span class="cl"><span class="sd">        -H &#34;Accept: application/vnd.github.v3+json&#34; \
</span></span></span><span class="line"><span class="cl"><span class="sd">        https://api.github.com/repos/fz42net/fz42net/dispatches \
</span></span></span><span class="line"><span class="cl"><span class="sd">        -d &#39;{&#34;event_type&#34;:&#34;resume_updated&#34;,&#34;client_payload&#34;:{&#34;version&#34;:&#34;${{ steps.get_version.outputs.version }}&#34;}}&#39;</span><span class="w">
</span></span></span></code></pre></div><p>The profile workflow listens for this:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">on</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">repository_dispatch</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">types</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="l">resume_updated]</span><span class="w">
</span></span></span></code></pre></div><p>Then it fetches the PDF from the private repo:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl">- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Download Latest Resume PDF</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">if</span><span class="p">:</span><span class="w"> </span><span class="l">github.event_name == &#39;repository_dispatch&#39; || github.event_name == &#39;workflow_dispatch&#39;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">      # Get the latest release from the private resume repo
</span></span></span><span class="line"><span class="cl"><span class="sd">      echo &#34;Fetching latest release info...&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">      LATEST_RELEASE=$(curl -s -H &#34;Authorization: token ${{ secrets.PAT_TOKEN }}&#34; \
</span></span></span><span class="line"><span class="cl"><span class="sd">        https://api.github.com/repos/fz42net/resume/releases/latest)
</span></span></span><span class="line"><span class="cl"><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">      # Check if we got a valid response
</span></span></span><span class="line"><span class="cl"><span class="sd">      if echo &#34;$LATEST_RELEASE&#34; | jq -e &#39;.assets&#39; &gt; /dev/null; then
</span></span></span><span class="line"><span class="cl"><span class="sd">        # Extract the download URL for resume.pdf
</span></span></span><span class="line"><span class="cl"><span class="sd">        DOWNLOAD_URL=$(echo &#34;$LATEST_RELEASE&#34; | jq -r &#39;.assets[] | select(.name==&#34;resume.pdf&#34;) | .url&#39;)
</span></span></span><span class="line"><span class="cl"><span class="sd">        
</span></span></span><span class="line"><span class="cl"><span class="sd">        if [ &#34;$DOWNLOAD_URL&#34; != &#34;null&#34; ] &amp;&amp; [ -n &#34;$DOWNLOAD_URL&#34; ]; then
</span></span></span><span class="line"><span class="cl"><span class="sd">          # Download the PDF
</span></span></span><span class="line"><span class="cl"><span class="sd">          curl -L -H &#34;Authorization: token ${{ secrets.PAT_TOKEN }}&#34; \
</span></span></span><span class="line"><span class="cl"><span class="sd">            -H &#34;Accept: application/octet-stream&#34; \
</span></span></span><span class="line"><span class="cl"><span class="sd">            &#34;$DOWNLOAD_URL&#34; -o resume.pdf
</span></span></span><span class="line"><span class="cl"><span class="sd">          
</span></span></span><span class="line"><span class="cl"><span class="sd">          echo &#34;Downloaded resume.pdf from version: ${{ github.event.client_payload.version || &#39;manual trigger&#39; }}&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">        else
</span></span></span><span class="line"><span class="cl"><span class="sd">          echo &#34;Error: Could not find resume.pdf in release assets&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">          exit 1
</span></span></span><span class="line"><span class="cl"><span class="sd">        fi
</span></span></span><span class="line"><span class="cl"><span class="sd">      else
</span></span></span><span class="line"><span class="cl"><span class="sd">        echo &#34;Error: Invalid API response or no releases found&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">        echo &#34;$LATEST_RELEASE&#34;
</span></span></span><span class="line"><span class="cl"><span class="sd">        exit 1
</span></span></span><span class="line"><span class="cl"><span class="sd">      fi</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">env</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">PAT_TOKEN</span><span class="p">:</span><span class="w"> </span><span class="l">${{ secrets.PAT_TOKEN }}</span><span class="w">
</span></span></span></code></pre></div><h2 id="the-result">The Result</h2>
<p>My profile now shows:</p>
<ul>
<li>Latest blog posts from fz42.net</li>
<li>Posts from my employer&rsquo;s blog</li>
<li>Link to current resume (from public repo)</li>
</ul>
<p><img loading="lazy" src="/posts/automating-my-github-profile-with-actions/images/github_profile.png" type="" alt="GitHub profile"  /></p>
<p>All updated automatically. The resume source stays private until I&rsquo;m ready to clean it up.</p>
<p>Full workflows: <a href="https://github.com/fz42net/fz42net/blob/main/.github/workflows/update-readme.yml">profile updater</a> (resume builder stays private for now)</p>
]]></content:encoded></item><item><title>Detecting Azure Front Door WAF with BChecks</title><link>https://fz42.net/posts/detecting-azure-front-door-waf-with-bcheck/</link><pubDate>Sun, 27 Jul 2025 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/detecting-azure-front-door-waf-with-bcheck/</guid><description>&lt;p&gt;TrustedSec &lt;a href="https://trustedsec.com/blog/azures-front-door-waf-wtf-ip-restriction-bypass"&gt;discovered&lt;/a&gt; that Azure Front Door WAF&amp;rsquo;s IP restrictions can be bypassed when using the default RemoteAddr setting. When the bypass works, ALL WAF rules get disabled - not just IP filtering.&lt;/p&gt;
&lt;p&gt;Front Door WAF has a specific fingerprint: 403 responses include both &lt;code&gt;x-azure-ref&lt;/code&gt; and &lt;code&gt;x-cache&lt;/code&gt; headers. Application Gateway WAF doesn&amp;rsquo;t include these.&lt;/p&gt;
&lt;p&gt;I&amp;rsquo;ll definitely forget these details in six months. That&amp;rsquo;s why I wrote this BCheck:&lt;/p&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-yaml" data-lang="yaml"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="nt"&gt;metadata&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;language&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="l"&gt;v2-beta&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;Azure Front Door WAF Detection&amp;#34;&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;Detects Azure Front Door WAF based on 403 response headers&amp;#34;&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;author&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;fz42&amp;#34;&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;tags&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;azure&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;waf&amp;#34;&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;bypass&amp;#34;&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt;&lt;/span&gt;&lt;span class="l"&gt;given response then&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="l"&gt;if &lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;{&lt;span class="l"&gt;latest.response.status_code} is &amp;#34;403&amp;#34; and &lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;{&lt;span class="l"&gt;latest.response.headers} matches &amp;#34;(?i)x-azure-ref&amp;#34; and &lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;{&lt;span class="l"&gt;latest.response.headers} matches &amp;#34;(?i)x-cache&amp;#34; &lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="l"&gt;then&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;report issue&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;severity&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="l"&gt;info&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;confidence&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="l"&gt;firm&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;detail&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;`Azure Front Door WAF detected. If using RemoteAddr (default) for IP restrictions, try X-Forwarded-For bypass&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;- &lt;span class="m"&gt;127.0.0.1&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;- &lt;span class="m"&gt;10.0.0.0&lt;/span&gt;&lt;span class="l"&gt;/8&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;- &lt;span class="m"&gt;172.16.0.0&lt;/span&gt;&lt;span class="l"&gt;/12&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;- &lt;span class="m"&gt;192.168.0.0&lt;/span&gt;&lt;span class="l"&gt;/16&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;- &lt;span class="l"&gt;...&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="l"&gt;If bypass works, ALL WAF protections are disabled.&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;References&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt;&lt;/span&gt;&lt;span class="l"&gt;https://trustedsec.com/blog/azures-front-door-waf-wtf-ip-restriction-bypass`&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;remediation&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;&amp;#34;Use SocketAddr instead of RemoteAddr or make a compound condition including both SocketAddr and RemoteAddr in WAF configuration.&amp;#34;&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="l"&gt;end if&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;p&gt;Now, I&amp;rsquo;ll have a nice reminder in Burp Suite ensuring I don&amp;rsquo;t miss this in the future.
&lt;img loading="lazy" src="https://fz42.net/posts/detecting-azure-front-door-waf-with-bcheck/images/bcheckissue.png" type="" alt="" /&gt;&lt;/p&gt;</description><content:encoded><![CDATA[<p>TrustedSec <a href="https://trustedsec.com/blog/azures-front-door-waf-wtf-ip-restriction-bypass">discovered</a> that Azure Front Door WAF&rsquo;s IP restrictions can be bypassed when using the default RemoteAddr setting. When the bypass works, ALL WAF rules get disabled - not just IP filtering.</p>
<p>Front Door WAF has a specific fingerprint: 403 responses include both <code>x-azure-ref</code> and <code>x-cache</code> headers. Application Gateway WAF doesn&rsquo;t include these.</p>
<p>I&rsquo;ll definitely forget these details in six months. That&rsquo;s why I wrote this BCheck:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">metadata</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">language</span><span class="p">:</span><span class="w"> </span><span class="l">v2-beta</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;Azure Front Door WAF Detection&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">description</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;Detects Azure Front Door WAF based on 403 response headers&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">author</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;fz42&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">tags</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;azure&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;waf&#34;</span><span class="p">,</span><span class="w"> </span><span class="s2">&#34;bypass&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="l">given response then</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="l">if </span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>{<span class="l">latest.response.status_code} is &#34;403&#34; and </span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>{<span class="l">latest.response.headers} matches &#34;(?i)x-azure-ref&#34; and </span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>{<span class="l">latest.response.headers} matches &#34;(?i)x-cache&#34; </span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="l">then</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		</span><span class="nt">report issue</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span><span class="nt">severity</span><span class="p">:</span><span class="w"> </span><span class="l">info</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span><span class="nt">confidence</span><span class="p">:</span><span class="w"> </span><span class="l">firm</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span><span class="nt">detail</span><span class="p">:</span><span class="w"> </span><span class="nt">`Azure Front Door WAF detected. If using RemoteAddr (default) for IP restrictions, try X-Forwarded-For bypass</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span>- <span class="m">127.0.0.1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span>- <span class="m">10.0.0.0</span><span class="l">/8</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span>- <span class="m">172.16.0.0</span><span class="l">/12</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span>- <span class="m">192.168.0.0</span><span class="l">/16</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span>- <span class="l">...</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            
</span></span></span><span class="line"><span class="cl"><span class="w">            </span><span class="l">If bypass works, ALL WAF protections are disabled.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            
</span></span></span><span class="line"><span class="cl"><span class="w">            </span><span class="nt">References</span><span class="p">:</span><span class="w"> 
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="l">https://trustedsec.com/blog/azures-front-door-waf-wtf-ip-restriction-bypass`</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span><span class="nt">remediation</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;Use SocketAddr instead of RemoteAddr or make a compound condition including both SocketAddr and RemoteAddr in WAF configuration.&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">		    
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="l">end if</span><span class="w">
</span></span></span></code></pre></div><p>Now, I&rsquo;ll have a nice reminder in Burp Suite ensuring I don&rsquo;t miss this in the future.
<img loading="lazy" src="/posts/detecting-azure-front-door-waf-with-bcheck/images/bcheckissue.png" type="" alt=""  /></p>
<h2 id="usage">Usage</h2>
<p>Drop this into Burp&rsquo;s BCheck editor. Every 403 response gets checked automatically.</p>
<h2 id="why-this-approach">Why this approach</h2>
<p>I&rsquo;ve collected 40+ bypass headers over the years (X-Forwarded-For, X-Real-IP, X-Client-IP, etc.). You can&rsquo;t blast all of them at every 403 on a client engagement - that&rsquo;s how you hit rate limits and get noticed.</p>
<p>This check identifies exactly when X-Forwarded-For will work (Front Door WAF with RemoteAddr). No spray and pray needed.</p>
<p>Between all the cloud providers and their quirks, I won&rsquo;t remember that x-azure-ref + x-cache = Front Door WAF six months from now. But my BCheck will.</p>
<p>Last week I hit 4 WAF-protected endpoints. If any had been vulnerable Front Door configurations, this would&rsquo;ve caught it automatically.</p>
]]></content:encoded></item><item><title>Justifying Claude Max with ccusage</title><link>https://fz42.net/posts/justifying-claude-max-with-ccusage/</link><pubDate>Mon, 21 Jul 2025 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/justifying-claude-max-with-ccusage/</guid><description>&lt;p&gt;I pay $200/month for Claude Max 10x and wanted to verify it&amp;rsquo;s actually saving me money.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://github.com/ryoppippi/ccusage"&gt;ccusage&lt;/a&gt; shows what I would have spent on API tokens:&lt;/p&gt;
&lt;p&gt;&lt;img loading="lazy" src="https://fz42.net/posts/justifying-claude-max-with-ccusage/images/ccusage_table.png" type="" alt="ccusage table" /&gt;&lt;/p&gt;
&lt;p&gt;Even in June (lighter usage), API tokens would have cost more than the subscription. July shows why I&amp;rsquo;ll keep paying - I&amp;rsquo;m already at 6x the subscription cost.&lt;/p&gt;
&lt;h2 id="the-numbers"&gt;The numbers&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Claude Max 5x: $100/month&lt;/li&gt;
&lt;li&gt;Claude Max 20x: $200/month&lt;/li&gt;
&lt;li&gt;My June API usage: $246&lt;/li&gt;
&lt;li&gt;My July API usage: $1,362 (and counting)&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="check-your-own-usage"&gt;Check your own usage&lt;/h2&gt;
&lt;div class="highlight"&gt;&lt;pre tabindex="0" class="chroma"&gt;&lt;code class="language-bash" data-lang="bash"&gt;&lt;span class="line"&gt;&lt;span class="cl"&gt;npx ccusage@latest monthly
&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;h2 id="context"&gt;Context&lt;/h2&gt;
&lt;p&gt;I run a side business and bill clients for development work, which makes a $200/month tool easier to justify. With a fixed monthly cost, I never think &amp;ldquo;is this prompt worth it?&amp;rdquo; or &amp;ldquo;should I try a different approach?&amp;rdquo; I just build. No mental math on whether refactoring this feature will cost $0.50 or $5.00 in tokens.&lt;/p&gt;</description><content:encoded><![CDATA[<p>I pay $200/month for Claude Max 10x and wanted to verify it&rsquo;s actually saving me money.</p>
<p><a href="https://github.com/ryoppippi/ccusage">ccusage</a> shows what I would have spent on API tokens:</p>
<p><img loading="lazy" src="/posts/justifying-claude-max-with-ccusage/images/ccusage_table.png" type="" alt="ccusage table"  /></p>
<p>Even in June (lighter usage), API tokens would have cost more than the subscription. July shows why I&rsquo;ll keep paying - I&rsquo;m already at 6x the subscription cost.</p>
<h2 id="the-numbers">The numbers</h2>
<ul>
<li>Claude Max 5x: $100/month</li>
<li>Claude Max 20x: $200/month</li>
<li>My June API usage: $246</li>
<li>My July API usage: $1,362 (and counting)</li>
</ul>
<h2 id="check-your-own-usage">Check your own usage</h2>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">npx ccusage@latest monthly
</span></span></code></pre></div><h2 id="context">Context</h2>
<p>I run a side business and bill clients for development work, which makes a $200/month tool easier to justify. With a fixed monthly cost, I never think &ldquo;is this prompt worth it?&rdquo; or &ldquo;should I try a different approach?&rdquo; I just build. No mental math on whether refactoring this feature will cost $0.50 or $5.00 in tokens.</p>
<p>That cognitive overhead disappeared with the subscription. The code ships faster when you&rsquo;re not rationing your tool usage.</p>
<p>For hobbyists, these numbers might not make sense. But if you&rsquo;re doing paid work, run ccusage and see where you land.</p>
]]></content:encoded></item><item><title>People aren't using IDEs anymore by end of year</title><link>https://fz42.net/posts/people-arent-using-ides-anymore-by-end-of-year/</link><pubDate>Tue, 15 Jul 2025 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/people-arent-using-ides-anymore-by-end-of-year/</guid><description>&lt;p&gt;I just watched &lt;a href="https://youtu.be/6eBSHbLKuN0?t=1575"&gt;Mastering Claude Code in 30 minutes&lt;/a&gt; by Boris Cherny from Anthropic, which is an excellent overview if you haven&amp;rsquo;t tried it yet. He demonstrates the capabilities clearly and shows some useful examples.&lt;/p&gt;
&lt;p&gt;But this prediction caught my attention:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&amp;ldquo;I think there&amp;rsquo;s a good chance that by the end of the year, people aren&amp;rsquo;t using IDEs anymore&amp;rdquo;&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;That&amp;rsquo;s a bold timeline.&lt;/p&gt;
&lt;p&gt;When I&amp;rsquo;m not doing security consulting, I&amp;rsquo;m building software and Claude Code has transformed my workflow. Whether it&amp;rsquo;s Python, Swift, TypeScript, C#, or Go, I can prototype ideas faster than I could ever code them manually. The ability to rapidly prototype and discard has fundamentally changed how I develop.&lt;/p&gt;</description><content:encoded><![CDATA[<p>I just watched <a href="https://youtu.be/6eBSHbLKuN0?t=1575">Mastering Claude Code in 30 minutes</a> by Boris Cherny from Anthropic, which is an excellent overview if you haven&rsquo;t tried it yet. He demonstrates the capabilities clearly and shows some useful examples.</p>
<p>But this prediction caught my attention:</p>
<blockquote>
<p>&ldquo;I think there&rsquo;s a good chance that by the end of the year, people aren&rsquo;t using IDEs anymore&rdquo;</p></blockquote>
<p>That&rsquo;s a bold timeline.</p>
<p>When I&rsquo;m not doing security consulting, I&rsquo;m building software and Claude Code has transformed my workflow. Whether it&rsquo;s Python, Swift, TypeScript, C#, or Go, I can prototype ideas faster than I could ever code them manually. The ability to rapidly prototype and discard has fundamentally changed how I develop.</p>
<p>But replacing IDEs entirely? By December 2025?</p>
<p>Will we see dramatic shifts in how we code? Absolutely. Are IDEs dead in 11 months? I&rsquo;ll take that bet.</p>
<p>The video is still worth your 30 minutes though. Boris gives a great demo of what Claude Code can do today.</p>
]]></content:encoded></item><item><title>Hello fz42</title><link>https://fz42.net/posts/hello-fz42/</link><pubDate>Tue, 01 Jul 2025 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/hello-fz42/</guid><description>Why I&amp;#39;m changing from fatzombi to fz42: zombie avatars don&amp;#39;t fit in client Slacks. Technical details on the domain setup and what&amp;#39;s staying the same.</description><content:encoded><![CDATA[<p>I&rsquo;m rebranding from fatzombi to fz42.</p>
<p>The main reason: professional contexts. Sharing a GitHub profile or email with &ldquo;fatzombi&rdquo; in client communications always felt off. The zombie avatar looked especially out of place in client Slack channels next to everyone&rsquo;s professional headshots. That disconnect matters when you&rsquo;re building trust as a security consultant.</p>
<p>fz42 solves this. It&rsquo;s short, professional, and easy to communicate verbally. The &ldquo;42&rdquo; is a Hitchhiker&rsquo;s Guide reference, because why not.</p>
<h2 id="technical-details">Technical details</h2>
<ul>
<li>Both fatzombi.com and fz42.net point to the same CloudFront distribution</li>
<li>Same S3 bucket serves both domains</li>
<li>RSS feeds work from either domain</li>
<li>No redirects needed—pick whichever URL you prefer</li>
</ul>
<p>The content remains the same—technical notes, tooling experiments, and whatever problems I&rsquo;m solving this week. Just with a handle that fits in a corporate Slack without raising eyebrows.</p>
]]></content:encoded></item><item><title>Migrating from IAM Secret and Access Keys to OIDC for Secure AWS Deployments</title><link>https://fz42.net/posts/migrate-iam-secrets-to-oidc/</link><pubDate>Wed, 29 May 2024 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/migrate-iam-secrets-to-oidc/</guid><description>Changing our GitHub Action to leverage OIDC instead of hard-coded AWS Secret and Access Keys.</description><content:encoded><![CDATA[<p>In our previous article (<a href="https://fz42.net/posts/jekyll-s3-githubactions/">Deploying a Jekyll website to AWS S3 with GitHub Actions and AWS CloudFormation</a>), we setup GitHub Actions to use IAM Secret and Access keys to deploy our blog to S3 and invalidate our CloudFront distribution cache.</p>
<p>This is the traditional approach to accessing AWS resources, but we’re going to look at migrating to <a href="https://openid.net/developers/how-connect-works/">OpenID Connect (OIDC)</a> to reduce our risk of credential exposure and simplify credential management.</p>
<h2 id="setting-up-oidc-in-aws">Setting up OIDC in AWS</h2>
<h3 id="create-an-oidc-provider-for-github">Create an OIDC provider for GitHub</h3>
<p>After navigating to the IAM dashboard in the AWS Console, click on <strong>Identity providers</strong>, then click <strong>Add provider</strong>.</p>
<p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/01-CreateIdentityProvider.png" type="" alt=""  /></p>
<p>We want to create an <strong>OpenID Connect</strong> provider with the following details:</p>
<ul>
<li><strong>Provider URL:</strong> <code>https://token.actions.githubusercontent.com</code></li>
<li><strong>Audience:</strong> <code>sts.amazonaws.com</code></li>
</ul>
<p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/02-AddIdentityProvider.png" type="" alt=""  /></p>
<p>Once redirected to the Identity Providers page, select your newly created provider and make note of its <strong>ARN</strong>.</p>
<p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/03-CopyARN.png" type="" alt=""  /></p>
<h3 id="create-an-iam-role">Create an IAM role</h3>
<p>At this point, we’ll create an IAM role that leverages this new identity provider using a trust policy and assign the minimum level of permissions to the role that we need.</p>
<p>Clicking the <strong>Assign role</strong> button under our identity provider will automatically populate the identity information in the dialog.</p>
<p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/04-AssignRole.png" type="" alt=""  /></p>
<p>We want to <strong>create a new role</strong>.</p>
<p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/05-CreateANewRole.png" type="" alt=""  /></p>
<p>At this screen, <strong>Web Identity</strong> and <strong>Identity Provider</strong> should automatically be selected. Ensure you select your <strong>Audience</strong> from the dropdown.</p>
<p>At the very least, you’ll need to enter the <strong>GitHub organization</strong>. If you have an individual account, the “organization” will be the name of your individual account. Additionally, if you want, you can limit the access of the identity to a specific repository and branch. I would recommend at least restricting to a specific repository.</p>
<p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/06-CreateWebIdentity.png" type="" alt=""  /></p>
<p>Click <strong>Next</strong> on the <strong>Add permissions</strong> step as we’ll create our own policy in a later step.</p>
<h4 id="scope-the-trust-policy">Scope the trust policy</h4>
<p>We can name our role and click the <strong>Create role</strong> button. The Trust policy should automatically be populated with the Identity Provider and GitHub organization/repo/branch information that was entered previously.</p>
<p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/07-CreateTrustPolicy.png" type="" alt=""  /></p>
<h4 id="assign-permissions">Assign permissions</h4>
<p>Navigate to your newly created IAM role.</p>
<p><em>Make note of the <strong>ARN</strong>, as that’s what we’ll need to provide to our GitHub Action.</em></p>
<p>Select <strong>Create inline policy</strong> under the <strong>Add permissions</strong> dropdown in the <strong>Permissions policies</strong> section.</p>
<p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/08-AssignPermissions.png" type="" alt=""  /></p>
<p>Clicking <strong>JSON</strong>, will bring us to the policy editor, where we can paste in the following policy that will limit our access to a specific S3 bucket and CloudFront distribution.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nt">&#34;Version&#34;</span><span class="p">:</span> <span class="s2">&#34;2012-10-17&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nt">&#34;Statement&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">        <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;Effect&#34;</span><span class="p">:</span> <span class="s2">&#34;Allow&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;Resource&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;arn:aws:s3:::fatzombi.com&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;arn:aws:s3:::fatzombi.com/*&#34;</span>
</span></span><span class="line"><span class="cl">            <span class="p">],</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;Action&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;s3:PutObject&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;s3:GetObject&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;s3:DeleteObject&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;s3:ListBucket&#34;</span>
</span></span><span class="line"><span class="cl">            <span class="p">]</span>
</span></span><span class="line"><span class="cl">        <span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;Effect&#34;</span><span class="p">:</span> <span class="s2">&#34;Allow&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;Resource&#34;</span><span class="p">:</span> <span class="s2">&#34;arn:aws:cloudfront::ACCOUNTID:distribution/DISTRIBUTION_ID&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;Action&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;cloudfront:CreateInvalidation&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;cloudfront:GetDistribution&#34;</span>
</span></span><span class="line"><span class="cl">                <span class="p">]</span>
</span></span><span class="line"><span class="cl">        <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/09-Permissions.png" type="" alt=""  /></p>
<p>Click <strong>Next</strong>, provide a name for our policy, then click <strong>Create policy</strong>.</p>
<p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/10-CreatePolicy.png" type="" alt=""  /></p>
<p>We now have a role with only the minimum level of access needed to deploy our files to S3 and invalidate the CloudFront cache.</p>
<h2 id="modifying-github-actions-workflow">Modifying GitHub Actions Workflow</h2>
<p>For reference, our original GitHub Action file consists of the following:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Deploy Website</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">on</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">workflow_dispatch</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">push</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">branches</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">main</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">jobs</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">build</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">runs-on</span><span class="p">:</span><span class="w"> </span><span class="l">ubuntu-latest</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">steps</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">actions/checkout@v3</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Build the site in the jekyll/builder container</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">        docker run \
</span></span></span><span class="line"><span class="cl"><span class="sd">        -v ${{ github.workspace }}:/srv/jekyll -v ${{ github.workspace }}/_site:/srv/jekyll/_site \
</span></span></span><span class="line"><span class="cl"><span class="sd">        jekyll/builder:latest /bin/bash -c &#34;chmod -R 777 /srv/jekyll &amp;&amp; jekyll build --future&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Configure AWS Credentials</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">aws-actions/configure-aws-credentials@v1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">with</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">aws-access-key-id</span><span class="p">:</span><span class="w"> </span><span class="l">${{ secrets.AWS_ACCESS_KEY }}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">aws-secret-access-key</span><span class="p">:</span><span class="w"> </span><span class="l">${{ secrets.AWS_SECRET_ACCESS_KEY }}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">aws-region</span><span class="p">:</span><span class="w"> </span><span class="l">us-east-1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Deploy to S3</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">        aws s3 sync _site s3://fatzombi.com --acl public-read --delete</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Invalidate CloudFront Cache</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">        aws cloudfront create-invalidation --distribution-id ${{ secrets.CLOUDFRONT_DISTRIBUTION }} --paths &#34;/*&#34;</span><span class="w">
</span></span></span></code></pre></div><p>The bulk of the changes with our new setup will be to the <strong>Configure AWS Credentials</strong> step, however we also need to add an additional block to our job so that it’s able to request an OIDC token and read our repository.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl">- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Configure AWS Credentials</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">aws-actions/configure-aws-credentials@v2</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">with</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">role-to-assume</span><span class="p">:</span><span class="w"> </span><span class="l">${{ secrets.AWS_IAM_ROLE_ARN }}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">role-session-name</span><span class="p">:</span><span class="w"> </span><span class="l">github-fatzombi-action</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">aws-region</span><span class="p">:</span><span class="w"> </span><span class="l">us-east-1</span><span class="w">
</span></span></span></code></pre></div><p>To summarize the changes of this step, we have:</p>
<ul>
<li>Upgraded to the @v2 library.</li>
<li>Removed the <code>aws-access-key-id</code> and <code>aws-secret-access-key</code> properties.</li>
<li>Added two new properties
<ul>
<li><strong>role-to-assume:</strong> which is the ARN of our IAM role we created previously.</li>
<li><strong>role-session-name:</strong> which is a unique identifier for the OIDC session that is created.</li>
</ul>
</li>
</ul>
<p>Lastly, we need to add a permissions block to our job, which grants the job permission to request an OIDC token and read our repository.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="w"> </span><span class="nt">permissions</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">id-token</span><span class="p">:</span><span class="w"> </span><span class="l">write</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span><span class="l">read</span><span class="w">
</span></span></span></code></pre></div><p>This bring our entire GitHub Action contents to the following:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Deploy Website</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">on</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">workflow_dispatch</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">push</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">branches</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">main</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">jobs</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">build</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">runs-on</span><span class="p">:</span><span class="w"> </span><span class="l">ubuntu-latest</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">permissions</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">id-token</span><span class="p">:</span><span class="w"> </span><span class="l">write</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">contents</span><span class="p">:</span><span class="w"> </span><span class="l">read</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">steps</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">actions/checkout@v3</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Build the site in the jekyll/builder container</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">        docker run \
</span></span></span><span class="line"><span class="cl"><span class="sd">        -v ${{ github.workspace }}:/srv/jekyll -v ${{ github.workspace }}/_site:/srv/jekyll/_site \
</span></span></span><span class="line"><span class="cl"><span class="sd">        jekyll/builder:latest /bin/bash -c &#34;chmod -R 777 /srv/jekyll &amp;&amp; jekyll build --future&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Configure AWS Credentials</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">aws-actions/configure-aws-credentials@v2</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">with</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">       </span><span class="nt">role-to-assume</span><span class="p">:</span><span class="w"> </span><span class="l">${{ vars.AWS_IAM_ROLE_ARN }}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">       </span><span class="nt">role-session-name</span><span class="p">:</span><span class="w"> </span><span class="l">github-fatzombi-action</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">       </span><span class="nt">aws-region</span><span class="p">:</span><span class="w"> </span><span class="l">us-east-1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Deploy to S3</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">        aws s3 sync _site s3://fatzombi --acl public-read --delete</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Invalidate CloudFront Cache</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">        aws cloudfront create-invalidation --distribution-id ${{ vars.CLOUDFRONT_DISTRIBUTION }} --paths &#34;/*&#34;</span><span class="w">
</span></span></span></code></pre></div><h2 id="testing-the-new-setup">Testing the new setup</h2>
<p>Commit the changes and we should see a successful deployment.</p>
<p><img loading="lazy" src="/posts/migrate-iam-secrets-to-oidc/images/11-Success.png" type="" alt=""  /></p>
<p>Now, you can go ahead and delete the <code>AWS_ACCESS_KEY</code> and <code>AWS_SECRET_ACCESS_KEY</code> repository secrets and delete the associated user in AWS, if it was solely used to deploy your blog.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Migrating from IAM secret and access keys to OpenID Connect (OIDC) for our GitHub Actions workflow helps enhance the security and management of our deployment.</p>
<p>By leveraging OIDC over the traditional IAM methods, we reduce the risk of credential exposure and simplify the process of managing access permissions.</p>
<p>I hope this guide has been helpful in your journey to modernize your AWS deployment practices. If you have any questions or feedback, feel free to reach out. Happy deploying!</p>
]]></content:encoded></item><item><title>Protect your S3-hosted static website with Origin Access Control</title><link>https://fz42.net/posts/protect-your-s3-hosted-static-website-with-origin-access-control/</link><pubDate>Thu, 23 May 2024 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/protect-your-s3-hosted-static-website-with-origin-access-control/</guid><description>&lt;p&gt;I previously wrote about
&lt;a href="https://fz42.net/posts/jekyll-s3-githubactions/"&gt;Deploying a Jekyll website to AWS S3 with GitHub Actions and AWS CloudFormation&lt;/a&gt;. However, as I continue to learn more about AWS, the more tweaks I realize we can make.&lt;/p&gt;
&lt;p&gt;What I want to walk through today is removing the public access to our S3 bucket which hosts our static site. We will configure an Origin Access Control to allow only CloudFront to access the S3 bucket. There&amp;rsquo;s one caveat to our use case, but we can resolve that using &lt;a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cloudfront-functions.html"&gt;CloudFront Functions&lt;/a&gt;.&lt;/p&gt;</description><content:encoded><![CDATA[<p>I previously wrote about
<a href="https://fz42.net/posts/jekyll-s3-githubactions/">Deploying a Jekyll website to AWS S3 with GitHub Actions and AWS CloudFormation</a>. However, as I continue to learn more about AWS, the more tweaks I realize we can make.</p>
<p>What I want to walk through today is removing the public access to our S3 bucket which hosts our static site. We will configure an Origin Access Control to allow only CloudFront to access the S3 bucket. There&rsquo;s one caveat to our use case, but we can resolve that using <a href="https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cloudfront-functions.html">CloudFront Functions</a>.</p>
<p>We&rsquo;ll be performing these steps through the AWS Console, but you could likely tweak the CloudFormation template that we created in the previous article.</p>
<h1 id="disclaimer"><strong>Disclaimer</strong></h1>
<p>The method we are implementing requires the use of a CloudFront function. The Always Free Tier includes 2 million function invocations per month. You should be able to determine cost based on the number of requests you are receiving per month, as each request will invoke our function.</p>
<p><img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image20.png" type="" alt=""  /></p>
<h1 id="adding-an-origin-access-control-policy-to-the-cloudfront-distribution">Adding an Origin Access Control policy to the CloudFront distribution</h1>
<p>Navigate to your Distribution and click <strong>Edit</strong> under the Settings.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image.png" type="" alt=""  />
Now go to <strong>Origins</strong> and click <strong>Edit</strong> on your existing origin.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image2.png" type="" alt=""  />
Scroll down and change <strong>Origin access</strong> from <strong>Public</strong> to <strong>Origin access control settings (recommended)</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image3.png" type="" alt=""  />
Now click <strong>Create new OAC</strong>. Leave all the values as their defaults and click <strong>Create</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image4.png" type="" alt=""  /></p>
<p>Make sure your newly created Origin access control is selected in the dropdown.</p>
<p>Click the <strong>Copy policy</strong> button.</p>
<p>Clicking the conveniently placed <strong>Go to S3 bucket permissions</strong> link will open a new tab, so don’t forgot to go back and click <strong>Save changes</strong> on this page.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image5.png" type="" alt=""  /></p>
<h1 id="updating-s3-bucket-policy">Updating S3 Bucket Policy</h1>
<p>Scroll down to the <strong>Bucket Policy</strong> and click the <strong>Edit</strong> button.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image6.png" type="" alt=""  />
Paste in the policy you copied from the previous step and click <strong>Save changes</strong>
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image7.png" type="" alt=""  />
At this point, we’ll be disabling both the static website hosting feature and public access on our S3 bucket.</p>
<h2 id="disabling-static-website-hosting">Disabling static website hosting</h2>
<p>Navigate to <strong>Properties</strong> of your S3 bucket and scroll down to <strong>Static website hosting</strong>, then click <strong>Edit</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image8.png" type="" alt=""  />
Go ahead and <strong>Disable</strong> the feature and click <strong>Save changes</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image9.png" type="" alt=""  /></p>
<h2 id="disabling-public-access">Disabling public access</h2>
<p>Navigate to the <strong>Permissions</strong> of your S3 bucket and scroll down to the <strong>Block public access (bucket settings)</strong>, then click <strong>Edit</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image10.png" type="" alt=""  />
Check <strong>Block all public access</strong> and click <strong>Save changes</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image11.png" type="" alt=""  />
Type <strong>confirm</strong> in the confirmation popup and click <strong>Confirm</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image12.png" type="" alt=""  /></p>
<p>Now there’s one caveat to this method. With static website hosting, if we had subdirectories on our website, they would be served properly. Unfortunately with this method, we receive an Access Denied page.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image13.png" type="" alt=""  /></p>
<p>Let’s fix that…</p>
<h1 id="setting-up-a-cloudfront-function">Setting up a CloudFront Function</h1>
<p>Navigate to the CloudFront dashboard and click <strong>Functions</strong>, then click <strong>Create function</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image14.png" type="" alt=""  /></p>
<p>Name your function, then leave the defaults selected, and click <strong>Create function</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image15.png" type="" alt=""  /></p>
<p>Scroll down to <strong>Function code</strong> and paste the following code in the editor. This code will append <code>index.html</code> to URIs, which will give us the desired behavior.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image16.png" type="" alt=""  /></p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-javascript" data-lang="javascript"><span class="line"><span class="cl"><span class="kd">function</span> <span class="nx">handler</span><span class="p">(</span><span class="nx">event</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="kd">var</span> <span class="nx">request</span> <span class="o">=</span> <span class="nx">event</span><span class="p">.</span><span class="nx">request</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="kd">var</span> <span class="nx">uri</span> <span class="o">=</span> <span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="k">if</span> <span class="p">(</span><span class="nx">uri</span><span class="p">.</span><span class="nx">endsWith</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">))</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="nx">request</span><span class="p">.</span><span class="nx">uri</span> <span class="o">+=</span> <span class="s1">&#39;index.html&#39;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">uri</span><span class="p">.</span><span class="nx">includes</span><span class="p">(</span><span class="s1">&#39;.&#39;</span><span class="p">))</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="nx">request</span><span class="p">.</span><span class="nx">uri</span> <span class="o">+=</span> <span class="s1">&#39;/index.html&#39;</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="nx">request</span><span class="p">;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><p>Click <strong>Save changes</strong>. Once you do this, you’ll see a gray pill labeled “Unpublished” next to the Publish tab. Click on <strong>Publish</strong>, then click <strong>Publish function</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image17.png" type="" alt=""  /></p>
<h2 id="attach-the-cloudfront-function-to-a-distribution">Attach the CloudFront Function to a Distribution</h2>
<p>Now, if you scroll down, you should see an <strong>Associated distributions</strong> section. Click the <strong>Add association</strong> button so that we can associate the function with our distribution.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image18.png" type="" alt=""  /></p>
<p>Pick your distribution from the Distribution dropdown list.</p>
<p>Select <strong>Viewer request</strong> from the Event type dropdown list.</p>
<p>Choose <strong>Default(*)</strong> as your cache behavior.</p>
<p>Now click <strong>Add association</strong>.
<img loading="lazy" src="/posts/protect-your-s3-hosted-static-website-with-origin-access-control/images/image19.png" type="" alt=""  /></p>
<p>That’s all there is to this. We’ve locked down our S3 bucket so that only CloudFront can access it.</p>
<p>An alternative to CloudFront functions would be to implement <a href="https://aws.amazon.com/lambda/edge/">Lambda@Edge</a>, however it would be more expensive (and overkill for our case) as you would pay for both the number of requests and the duration that your function runs.</p>
]]></content:encoded></item><item><title>Configuring a Private Burp Collaborator on AWS EC2 with Route 53 and Let's Encrypt</title><link>https://fz42.net/posts/private_burp_collaborator/</link><pubDate>Mon, 23 Oct 2023 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/private_burp_collaborator/</guid><description>Tutorial demonstrating how to configure a private Burp Collaborator instance on AWS EC2 behind AWS Route 53 and Let&amp;#39;s Encrypt for TLS certificates.</description><content:encoded><![CDATA[<h1 id="introduction">Introduction</h1>
<p>As application security testers, we encounter scenarios where clients blacklist the default Burp Collaborator domains due to security and privacy concerns. Frequent enough that it warrants the configuration of a private Burp Collaborator server.</p>
<p>In this article, I will guide you through the process of configuring a private Burp Collaborator instance on an AWS Elastic Compute Cloud (EC2) instance while integrating Let&rsquo;s Encrypt for SSL certificate security. I&rsquo;ll conclude by validating the environment with a test in Burp Suite.</p>
<p>Let&rsquo;s proceed!</p>
<h1 id="prerequisites">Prerequisites</h1>
<p>Before I begin, ensure you have the following prerequisites in place:</p>
<ul>
<li><strong>AWS Account:</strong> You need an active Amazon Web Services (AWS) account.</li>
<li><strong>Hosted Zone in AWS Route 53:</strong> Your domain doesn&rsquo;t need to be registered through AWS, but you need to manage DNS for the domain through Route 53.</li>
<li><strong>Burp Suite Professional:</strong> You&rsquo;ll need the professional or enterprise version to run a collaborator instance.</li>
</ul>
<h1 id="overview">Overview</h1>
<p>For this tutorial, we will configure Burp Collaborator to run on a subdomain:</p>
<p><strong>Domain:</strong> <code>fatzombi.com</code></p>
<p><strong>Subdomain:</strong> <code>burp.fatzombi.com</code></p>
<p>To maintain transparency, redacted screenshots won&rsquo;t be featured, except for masking my home IP address. The subdomain and EC2 instance will be terminated after publishing.</p>
<p>Disclaimer: Please be aware that using AWS Route 53 and EC2 services may result in charges. AWS pricing is usage-based, and data transfer costs, resource usage, and data storage can all contribute to your expenses. It is essential to monitor and manage your usage to avoid unexpected costs.</p>
<h1 id="launching-an-ec2instance">Launching an EC2 instance</h1>
<p>This section guides you through setting up a new Amazon Elastic Compute Cloud (EC2) instance. This instance will serve as the foundation for your private Burp Collaborator setup.</p>
<ol>
<li>Log in to the AWS Management Console</li>
<li>Search for EC2 under the compute section of the AWS services search bar.</li>
<li>Click on Instances in the EC2 dashboard.</li>
<li>Click on the Launch Instances button in the top-right corner of the Instances page.</li>
</ol>
<p>Now you can define options for the EC2 VM.</p>
<p><strong>Name:</strong> Burp Collaborator</p>
<p><strong>Application and OS Images:</strong> Ubuntu 22.04 LTS 64-bit (x86)</p>
<p><strong>Instance Type:</strong> t2.micro for the sake of this tutorial</p>
<p><strong>Key pair (login):</strong> Select an existing key pair or click the Create new key pair button to upload the SSH key you&rsquo;ll use to access the VM.</p>
<p><strong>Network Settings, Storage, and Advanced Details:</strong> Defaults are fine for the sake of this tutorial.</p>
<p>Click on Launch Instance, and within a few seconds, you can access the instance by clicking on the instance ID.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*D8nOrnMZ6wBKkzS_oyd2mw.webp" type="" alt=""  /></p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*H1wzDWw-GvOBFqQQPCUDCA.webp" type="" alt=""  /></p>
<p>Make note of the <strong>Public IPv4 address</strong> and <strong>Private IPv4 addresses</strong> for Burp Collaborator and DNS configurations.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*N7-NvBpLXpLy9oqg37203A.webp" type="" alt=""  /></p>
<p>Since the configuration happens over SSH, confirm access to the VM via its Public IPv4 address.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">ssh ubuntu@54.82.28.144
</span></span></code></pre></div><p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*Kaar7WND9kQnPo6bElmc2w.webp" type="" alt=""  /></p>
<h2 id="configuring-inbound-ports-to-ec2">Configuring Inbound Ports to EC2</h2>
<p>Burp Collaborator requires specific ports to be open on your EC2 VM. In the EC2 instance summary, navigate to the Security tab and click on the Security group.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*t_EMQR_JyydjsSc_0f4PmA.webp" type="" alt=""  /></p>
<p>Clicking on the Edit inbound rules button will allow us to enter more rules than the default configuration.
Add the following nine rules with a source of Anywhere-IPv4.</p>
<ul>
<li>TCP/80</li>
<li>TCP/443</li>
<li>TCP/25</li>
<li>TCP/587</li>
<li>TCP/465</li>
<li>TCP/9090</li>
<li>TCP/9443</li>
<li>TCP/53</li>
<li>UDP/53</li>
</ul>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*CkripQJXdAdXhb8-UL2AFw.webp" type="" alt=""  /></p>
<h2 id="configuring-dns-records-with-route-53">Configuring DNS records with Route 53</h2>
<p>As previously mentioned, we’re configuring Burp Collaborator to run on a subdomain.</p>
<p><strong>Domain:</strong> <code>fatzombi.com</code></p>
<p><strong>Subdomain:</strong> <code>burp.fatzombi.com</code></p>
<p>Access the Hosted Zone in the AWS Management Console for the domain.</p>
<p>Create an <strong>A</strong> record for <code>burp.fatzombi.com</code> pointing to the Public IPv4 address of your EC2 VM.</p>
<p>Click the <strong>Create record</strong> button under the hosted zone.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*f9F9ojdehVMES55bd-7qtA.webp" type="" alt=""  /></p>
<p>For now, that’s all the DNS records we need to add. We’ll add a <strong>TXT</strong> record in the next step, then an <strong>NS</strong> and <strong>A</strong> record in the following step; otherwise Let’s Encrypt won’t successfully generate the certificate.</p>
<p>To confirm DNS propagation, access your EC2 via its subdomain, bearing in mind that DNS propagation may take time. You can use a website such as <a href="https://www.whatsmydns.net/">WhatsMyDns.net</a> to monitor the progress.</p>
<h2 id="configuring-lets-encrypt-ssl-certificate">Configuring Let’s Encrypt SSL Certificate</h2>
<p>To generate the Let’s Encrypt certificate, install <code>certbot</code> via <code>snapd</code>.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">sudo apt update
</span></span><span class="line"><span class="cl">sudo apt install snapd
</span></span><span class="line"><span class="cl">sudo snap install certbot --classic
</span></span><span class="line"><span class="cl">sudo ln -s /snap/bin/certbot /usr/bin/certbot
</span></span></code></pre></div><p>Generate a certificate, customizing the <code>-m</code> and <code>-d</code> parameters with your email address and domain respectively.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">sudo certbot certonly -m phil@fatzombi.com <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span>-d <span class="s2">&#34;burp.fatzombi.com,*.burp.fatzombi.com&#34;</span> <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span>--server https://acme-v02.api.letsencrypt.org/directory <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span>--manual --agree-tos --no-eff-email --preferred-challenges dns-01
</span></span></code></pre></div><p>This command’s output will instruct you to create a DNS TXT record with a specific name and value.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*tkqUmvsU0vES0mKMs4eV7A.webp" type="" alt=""  /></p>
<p>Update Route 53 with this record. Set a low Time to Live (TTL) since you’ll be modifying this value.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*Kawbvy6ERU3Oa1YtHHiSIA.webp" type="" alt=""  /></p>
<p>You’ll note that you don’t want to copy and paste the record name verbatim, or it will duplicate your domain suffix and Let’s Encrypt won’t detect the record.</p>
<p>Wait a few minutes or use a tool such as Dig under the <a href="https://toolbox.googleapps.com/apps/dig/#TXT/">Google Admin Toolbox</a> to validate the TXT record, which returns the value you just created.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*4y31wMc0xjlES_EndfA1PQ.webp" type="" alt=""  /></p>
<p>At this point, click Enter in your terminal, where you’ll be prompted to enter another DNS TXT record.</p>
<p>However, it’ll have the same record name, but we can’t add two records with the same name.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*qXNc4WtLuOMZ1AjCHT6jMA.webp" type="" alt=""  /></p>
<p>What you need to do is edit the existing <strong>TXT</strong> record and add the additional value on a new line. Make sure the values are separated by a line break, or the validation will fail.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*VxPE3JPsTrZ-u-sOBAFN0w.webp" type="" alt=""  /></p>
<p>Confirm both values are present using a tool like the Google Admin Toolbox.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*91_9t8q2hD2JMtRvtM7xNA.webp" type="" alt=""  /></p>
<p>With these steps complete, you should have a certificate and key file saved to your system.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*C97N8yu4aBPzOp3OAs6kyA.webp" type="" alt=""  /></p>
<h2 id="allow-collaborator-to-be-authoritative-for-your-subdomain">Allow Collaborator to be Authoritative for your subdomain</h2>
<p>Burp Collaborator will generate random subdomains under your domain. To enable this, set the Collaborator server as authoritative for the subdomain.</p>
<p>Add two records to your hosted zone in Route 53:</p>
<p><strong>NS record:</strong> Ensures that queries for the subdomain are resolved by your collaborator server.</p>
<p><strong>A record:</strong> Resolves your name server to the Public IPv4 address of your collaborator server.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*DHwpxbc-HNYkzpUGJiR_1g.webp" type="" alt=""  /></p>
<p>The next record will ensure that <code>ns1.burp.fatzombi.com</code> resolves to your EC2 server.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*TvWfODGdEwX9hM7KPlKMNA.webp" type="" alt=""  /></p>
<h1 id="install-configure-and-run-burp-collaborator">Install, Configure, and Run Burp Collaborator</h1>
<p>At this stage, with all your DNS records configured, it’s time to install, configure, and run Burp Collaborator.</p>
<h3 id="installing-burp-suite-professional">Installing Burp Suite Professional</h3>
<p>Log in to your PortSwigger dashboard and download the Linux version of Burp Suite.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*dbjNTkKRIfhO1WvykRyTkw.webp" type="" alt=""  /></p>
<p>Transfer the file to your EC2:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">scp burpsuite_pro_linux_v2023_10_2_2.sh ubuntu@54.82.28.144:/home/ubuntu/
</span></span></code></pre></div><p>Return to your EC2 VM and make the file executable:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">sudo chmod u+x burpsuite_pro_linux_v2023_10_2_2.sh
</span></span></code></pre></div><p>Now, proceed to install Burp Suite, which mainly involves accepting the default prompts:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">sudo ./burpsuite_pro_linux_v2023_10_2_2.sh
</span></span></code></pre></div><p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*GdKxdPNEL7jpagJTnl7Rhw.webp" type="" alt=""  /></p>
<h3 id="configuring-collaborator">Configuring Collaborator</h3>
<p>For storing your configuration and miscellaneous files, create a directory under <code>/usr/local/collaborator</code>.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">sudo mkdir /usr/local/collaborator
</span></span><span class="line"><span class="cl"><span class="nb">cd</span> /usr/local/collaborator
</span></span></code></pre></div><h4 id="contents-of-collaboratorconfig">Contents of collaborator.config</h4>
<p>I won’t delve into the details of the configuration, as PortSwigger provides the <a href="https://portswigger.net/burp/documentation/collaborator/server/private/example">collaborator.config example</a> with comprehensive property explanations. However, the values you need to modify are as follows:</p>
<ul>
<li>serverDomain</li>
<li>eventCapture.localAddress</li>
<li>eventCapture.publicAddress</li>
<li>eventCapture.ssl.certificateFiles</li>
<li>poling.localAddress</li>
<li>polling.publicAddress</li>
<li>polling.ssl.certificateFiles</li>
<li>dns.interfaces.name</li>
<li>dns.interfaces.localAddress</li>
<li>dns.interfaces.publicAddress</li>
</ul>
<p>Save the file with the contents under <code>/usr/local/collaborator/collaborator.config</code>.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-json" data-lang="json"><span class="line"><span class="cl"><span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nt">&#34;serverDomain&#34;</span><span class="p">:</span> <span class="s2">&#34;burp.fatzombi.com&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nt">&#34;workerThreads&#34;</span><span class="p">:</span> <span class="mi">10</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="nt">&#34;eventCapture&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;localAddress&#34;</span><span class="p">:</span> <span class="s2">&#34;172.31.8.236&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;publicAddress&#34;</span><span class="p">:</span> <span class="s2">&#34;54.82.28.144&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;http&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;ports&#34;</span><span class="p">:</span> <span class="mi">80</span>
</span></span><span class="line"><span class="cl">        <span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;https&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;ports&#34;</span><span class="p">:</span> <span class="mi">443</span>
</span></span><span class="line"><span class="cl">        <span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;smtp&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;ports&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">                <span class="mi">25</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="mi">587</span>
</span></span><span class="line"><span class="cl">            <span class="p">]</span>
</span></span><span class="line"><span class="cl">        <span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;smtps&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;ports&#34;</span><span class="p">:</span> <span class="mi">465</span>
</span></span><span class="line"><span class="cl">        <span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;ssl&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;certificateFiles&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;/etc/letsencrypt/live/burp.fatzombi.com/privkey.pem&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;/etc/letsencrypt/live/burp.fatzombi.com/cert.pem&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;/etc/letsencrypt/live/burp.fatzombi.com/fullchain.pem&#34;</span>
</span></span><span class="line"><span class="cl">            <span class="p">]</span>
</span></span><span class="line"><span class="cl">        <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">},</span>
</span></span><span class="line"><span class="cl">    <span class="nt">&#34;polling&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;localAddress&#34;</span><span class="p">:</span> <span class="s2">&#34;172.31.8.236&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;publicAddress&#34;</span><span class="p">:</span> <span class="s2">&#34;54.82.28.144&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;http&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;port&#34;</span><span class="p">:</span> <span class="mi">9090</span>
</span></span><span class="line"><span class="cl">        <span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;https&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;port&#34;</span><span class="p">:</span> <span class="mi">9443</span>
</span></span><span class="line"><span class="cl">        <span class="p">},</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;ssl&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;certificateFiles&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;/etc/letsencrypt/live/burp.fatzombi.com/privkey.pem&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;/etc/letsencrypt/live/burp.fatzombi.com/cert.pem&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">                <span class="s2">&#34;/etc/letsencrypt/live/burp.fatzombi.com/fullchain.pem&#34;</span>
</span></span><span class="line"><span class="cl">            <span class="p">]</span>
</span></span><span class="line"><span class="cl">        <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">},</span>
</span></span><span class="line"><span class="cl">    <span class="nt">&#34;metrics&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;path&#34;</span><span class="p">:</span> <span class="s2">&#34;metrics-ss&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">},</span>
</span></span><span class="line"><span class="cl">    <span class="nt">&#34;dns&#34;</span><span class="p">:</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;interfaces&#34;</span><span class="p">:</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">            <span class="p">{</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;name&#34;</span><span class="p">:</span> <span class="s2">&#34;ns1.burp.fatzombi.com&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;localAddress&#34;</span><span class="p">:</span> <span class="s2">&#34;172.31.8.236&#34;</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">            <span class="nt">&#34;publicAddress&#34;</span><span class="p">:</span> <span class="s2">&#34;54.82.28.144&#34;</span>
</span></span><span class="line"><span class="cl">            <span class="p">}</span>
</span></span><span class="line"><span class="cl">        <span class="p">],</span>
</span></span><span class="line"><span class="cl">        <span class="nt">&#34;ports&#34;</span><span class="p">:</span> <span class="mi">53</span>
</span></span><span class="line"><span class="cl">    <span class="p">},</span>
</span></span><span class="line"><span class="cl">    <span class="nt">&#34;logLevel&#34;</span><span class="p">:</span> <span class="s2">&#34;INFO&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><h2 id="running-collaborator">Running Collaborator</h2>
<p>With a working configuration, you can now launch Collaborator.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">/usr/local/BurpSuitePro/BurpSuitePro -Xmx200m --collaborator-server --collaborator-config<span class="o">=</span>/usr/local/collaborator/collaborator.config
</span></span></code></pre></div><p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*YWm8W2UjMyEaWpjkOQ0vSw.webp" type="" alt=""  /></p>
<h1 id="testing-with-burp-suite">Testing with Burp Suite</h1>
<p>Return to your desktop, launch Burp Suite Professional, and navigate to the Collaborator tab under Settings -&gt; Project.
Modify the <strong>Server Location</strong> and <strong>Polling location</strong>, noting that the latter includes port 9443.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*81Kl_9ZruT8fSnfeYj-Afg.webp" type="" alt=""  /></p>
<p>Click the <strong>Run health check</strong>&hellip; button to verify your setup, you should observe a series of green successes.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*YKU307PYIxJeKsdgxWdbDw.webp" type="" alt=""  /></p>
<p>Send a payload containing the Collaborator-generated URL, and we can see the HTTP and DNS requests in the Collaborator tab.</p>
<p><img loading="lazy" src="/posts/private_burp_collaborator/images/1*1d-rc_uwG_hPcCmrs4tCBg.webp" type="" alt=""  /></p>
<h1 id="taking-it-further">Taking it further</h1>
<p>As of now, your Burp Collaborator is operational, but you can take additional steps:</p>
<ul>
<li>Create a service file for systemd to ensure Burp Collaborator starts on boot.</li>
<li>Set up an Elastic IP on the EC2 VM to prevent changes in your Public IPv4 address during reboots.</li>
<li>Configure a cronjob for Let’s Encrypt certificate renewal.</li>
<li>Run Collaborator as a lower-privileged user instead of root.</li>
</ul>
<h1 id="conclusion">Conclusion</h1>
<p>In the world of application security testing, the ability to adapt to the unique requirements and constraints of your clients is a hallmark of a skilled tester. Often, this means dealing with clients who have stringent security policies that necessitate bypassing default settings, such as the use of the standard Burp Collaborator domains.</p>
<p>Through this tutorial, I’ve equipped you with the knowledge to overcome such challenges and create a private and secure Burp Collaborator on an Amazon Elastic Compute Cloud (EC2) instance within AWS.</p>
<p>Now, it’s your turn to strengthen your cybersecurity arsenal and enhance the security posture of the web applications you assess. Embrace the power of a private Burp Collaborator, and lead the charge in securing the digital world, one application at a time.</p>
<p>Happy testing!</p>
]]></content:encoded></item><item><title>Automating Your Homelab with Proxmox, Cloud-init, Terraform, and Ansible — Part 3- Automating with Ansible</title><link>https://fz42.net/posts/automating-your-homelab-p3-ansible/</link><pubDate>Sun, 28 May 2023 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/automating-your-homelab-p3-ansible/</guid><description>&lt;p&gt;Picking back up where we left off in &lt;a href="https://fz42.net/posts/automating-your-homelab-p2-terraform/"&gt;Part 2&lt;/a&gt; of our homelab automation series, we now have Terraform creating our minimally configured VMs. However, installing and configuring software and settings on VMs can still be a tedious and time-consuming task, especially if you have many VMs and services to manage. This is where Ansible comes in.&lt;/p&gt;
&lt;h2 id="overview-of-ansible"&gt;Overview of Ansible&lt;/h2&gt;
&lt;p&gt;Ansible is an open source infrastructure as code tool that allows you to provision software, perform configuration management, and handle application deployments. Similar to Terraform, it uses a declarative language that is simple to write. Additionally, there are thousands of modules, most can be found at &lt;del&gt;&lt;a href="https://docs.ansible.com/ansible/latest/collections/index_module.html"&gt;https://docs.ansible.com/ansible/latest/collections/index_module.html&lt;/a&gt;&lt;/del&gt;. In this article, we will explore how to use Ansible to automate the configuration of our VMs.&lt;/p&gt;</description><content:encoded><![CDATA[<p>Picking back up where we left off in <a href="https://fz42.net/posts/automating-your-homelab-p2-terraform/">Part 2</a> of our homelab automation series, we now have Terraform creating our minimally configured VMs. However, installing and configuring software and settings on VMs can still be a tedious and time-consuming task, especially if you have many VMs and services to manage. This is where Ansible comes in.</p>
<h2 id="overview-of-ansible">Overview of Ansible</h2>
<p>Ansible is an open source infrastructure as code tool that allows you to provision software, perform configuration management, and handle application deployments. Similar to Terraform, it uses a declarative language that is simple to write. Additionally, there are thousands of modules, most can be found at <del><a href="https://docs.ansible.com/ansible/latest/collections/index_module.html">https://docs.ansible.com/ansible/latest/collections/index_module.html</a></del>. In this article, we will explore how to use Ansible to automate the configuration of our VMs.</p>
<p>If you want to skip ahead, you can find the Ansible playbook over at <del><a href="https://github.com/fatzombi/ansible-homelab">https://github.com/fatzombi/ansible-homelab</a></del>.</p>
<h2 id="our-current-vm-setup">Our Current VM setup</h2>
<p>If you’re following along through all the parts of this series, there is one modification that I made since the previous article. This change replaces the HomeBridge VM with a new host with the sole purpose of running docker contains, since that is very popular inside homelabs. You can either browse the repository listed above or copy the <code>pihole.tf</code> file and save it as <code>docker.tf</code> and make the necessary modifications <code>todocker.tf</code> and <code>vars.tf</code> file.</p>
<p>Additionally, if you’re following along with this article, you may see a 10.0.5.x network referenced in screenshots, due to a collision on my production network. Wherever you see this, assume the network is 192.168.1.x per our configuration.</p>
<h2 id="configuring-ansible-on-proxmox">Configuring Ansible on Proxmox</h2>
<p>Proxmox doesn’t ship with Ansible, but it is in the Debian repositories. We can quickly install it by running the following:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">apt install ansible
</span></span></code></pre></div><p>Now, let’s create a directory that will contain our Ansible playbook and configuration, ensuring we create it as a git repository so that we can track changes over time.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">mkdir ansible-homelab
</span></span><span class="line"><span class="cl"><span class="nb">cd</span> ansible-homelab
</span></span><span class="line"><span class="cl">git init
</span></span></code></pre></div><p>OR, if you like, you can clone the entire playbook.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">git clone https://github.com/fatzombi/ansible-homelab
</span></span><span class="line"><span class="cl"><span class="nb">cd</span> ansible-homelab
</span></span></code></pre></div><h2 id="creating-an-ansible-playbook">Creating an Ansible playbook</h2>
<ol>
<li>Define our inventory. This is the collection and grouping of hosts which we want to execute our playbook against.</li>
<li>Configure settings for how Ansible will talk to each of the hosts.</li>
<li>Create a playbook that defines the relationships between our hosts and the roles we will define in the next step.</li>
<li>Create roles for our servers.
a. Debian role, this is our base role that we want applied to all our servers.
b. PiHole role, this is a service specific role that will configure our PiHole server.
c. Docker role, this is a service specific role that will configure our Docker host.</li>
</ol>
<h3 id="the-inventory">The Inventory</h3>
<p>First off, we need to define the inventory of our hosts that Ansible will run tasks against. While there are modules which allow a dynamic inventory to be provided by Terraform, we will be statically defining our inventory for this article.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl"><span class="nb">echo</span> -e <span class="s2">&#34;[pihole]\n192.168.1.210\n&#34;</span> &gt;&gt; hosts
</span></span><span class="line"><span class="cl"><span class="nb">echo</span> -e <span class="s2">&#34;[docker]\n192.168.1.212\n&#34;</span> &gt;&gt; hosts
</span></span></code></pre></div><p><img loading="lazy" src="/posts/automating-your-homelab-p3-ansible/images/1*pAYW8sDR4WzeC2MAxBGnmQ.webp" type="" alt=""  /></p>
<h3 id="ansible-configuration">Ansible Configuration</h3>
<p>Based on our setup, we need to modify a couple of settings that Ansible will use to communicate with our servers.</p>
<p>Ansible will read files from a <code>group_vars</code> directory. These files are where we can define variables which can be used throughout our playbook. The variables can be applied to all hosts, or we can associate the variables with specific roles. For example, we can define <code>DNS_SERVER_1</code> for all hosts, and we could define <code>SQL_DEFAULT_PASSWORD</code> specifically for hosts that inherit the database role of database.</p>
<p>For this article, we will apply a minimal configuration, such that Ansible uses Python3 and SSH.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">mkdir group_vars
</span></span><span class="line"><span class="cl"><span class="nb">echo</span> <span class="s2">&#34;ansible_connection: ssh&#34;</span> &gt;&gt; group_vars/all
</span></span><span class="line"><span class="cl"><span class="nb">echo</span> <span class="s2">&#34;ansible_python_interpreter: /usr/bin/python3&#34;</span> &gt;&gt; group_vars/all
</span></span></code></pre></div><h3 id="defining-our-playbook">Defining our Playbook</h3>
<p>For simplicity, we will have a Debian role, which is applied to all of our servers, then we will define individual roles for each additional server. Notice, we can make use of the all value for the hosts definition in our first task.</p>
<p>To achieve this, we need to create the following contents in a <code>playbook.yml</code> file.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nn">---</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Ansible playbook for all our hosts</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">hosts</span><span class="p">:</span><span class="w"> </span><span class="l">all</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">roles</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">remote_user</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">become</span><span class="p">:</span><span class="w"> </span><span class="kc">yes</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Ansible playbook for configuring pihole</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">hosts</span><span class="p">:</span><span class="w"> </span><span class="l">pihole</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">roles</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="l">pihole</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">remote_user</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">become</span><span class="p">:</span><span class="w"> </span><span class="kc">yes</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Ansible playbook for configuring homebridge</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">hosts</span><span class="p">:</span><span class="w"> </span><span class="l">homebridge</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">roles</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="l">homebridge</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">remote_user</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">become</span><span class="p">:</span><span class="w"> </span><span class="kc">yes</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Ansible playbook for configuring docker</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">hosts</span><span class="p">:</span><span class="w"> </span><span class="l">docker</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">roles</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="l">docker</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">remote_user</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">become</span><span class="p">:</span><span class="w"> </span><span class="kc">yes</span><span class="w">
</span></span></span></code></pre></div><h3 id="configuring-tasks">Configuring tasks</h3>
<h4 id="debian-tasks">Debian tasks</h4>
<p>Often there is software which you’d like installed on all your hosts. While this could have been handled inside the base cloud-init image, it isn’t always the best option for several potential reasons.</p>
<p>Therefore, this will be the core function of our <strong>debian</strong> role. We want to update <code>apt</code> and install a few packages across all our hosts.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">mkdir -p roles/debian/tasks
</span></span></code></pre></div><p>Now we can place the following contents inside <code>roles/debian/tasks/main.yml</code>.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nn">---</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s1">&#39;Update APT package cache&#39;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">apt</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">update_cache</span><span class="p">:</span><span class="w"> </span><span class="kc">yes</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">upgrade</span><span class="p">:</span><span class="w"> </span><span class="l">safe</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Install packages</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">apt</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">package</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">git</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">apt-transport-https</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">ca-certificates</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">wget</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">software-properties-common</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">gnupg2</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">curl</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">python3-pip</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">state</span><span class="p">:</span><span class="w"> </span><span class="l">present</span><span class="w">
</span></span></span></code></pre></div><h4 id="pihole-tasks">Pihole tasks</h4>
<p>Whether you’re familiar with the PiHole installation process, there are several steps that we must take.</p>
<ol>
<li>Cron must be installed on our host.</li>
<li>We need to clone the PiHole repo.</li>
<li>Create a setupVars.conf configuration file.</li>
<li>Install PiHole.</li>
<li>Reboot PiHole.</li>
<li>Then we’d go ahead and configure our DNS to use PiHole.</li>
</ol>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">mkdir -p roles/pihole/tasks/
</span></span></code></pre></div><p>Now we can place the following contents inside roles<code>/pihole/tasks/main.yml</code>.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">---
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">- name: Install packages
</span></span><span class="line"><span class="cl">  apt:
</span></span><span class="line"><span class="cl">    package: 
</span></span><span class="line"><span class="cl">      - cron
</span></span><span class="line"><span class="cl">      - git
</span></span><span class="line"><span class="cl">    state: present
</span></span><span class="line"><span class="cl">  tags: pihole
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">- name: <span class="s1">&#39;Clone pihole repo&#39;</span>
</span></span><span class="line"><span class="cl">  ansible.builtin.git:
</span></span><span class="line"><span class="cl">    repo: <span class="s1">&#39;https://github.com/pi-hole/pi-hole&#39;</span>
</span></span><span class="line"><span class="cl">    dest: /home/debian/pi-hole
</span></span><span class="line"><span class="cl">  tags: pihole
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">- name: <span class="s1">&#39;Create pihole config directory&#39;</span>
</span></span><span class="line"><span class="cl">  file: <span class="nv">path</span><span class="o">=</span>/etc/pihole/ <span class="nv">state</span><span class="o">=</span>directory
</span></span><span class="line"><span class="cl">  tags: pihole
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">- name: <span class="s1">&#39;Copy setupVars.conf&#39;</span>
</span></span><span class="line"><span class="cl">  copy: <span class="nv">src</span><span class="o">=</span>templates/setupVars.conf.j2 <span class="nv">dest</span><span class="o">=</span>/etc/pihole/setupVars.conf
</span></span><span class="line"><span class="cl">  tags: pihole
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">- name: <span class="s1">&#39;Install pi-hole&#39;</span>
</span></span><span class="line"><span class="cl">  ansible.builtin.shell: ./basic-install.sh --unattended
</span></span><span class="line"><span class="cl">  args:
</span></span><span class="line"><span class="cl">    chdir: <span class="s2">&#34;/home/debian/pi-hole/automated install&#34;</span>
</span></span><span class="line"><span class="cl">  tags: pihole
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">- name: <span class="s1">&#39;Reboot&#39;</span>
</span></span><span class="line"><span class="cl">  shell: sleep <span class="m">2</span> <span class="o">&amp;&amp;</span> reboot
</span></span><span class="line"><span class="cl">  async: <span class="m">1</span>
</span></span><span class="line"><span class="cl">  poll: <span class="m">0</span>
</span></span><span class="line"><span class="cl">  ignore_errors: <span class="nb">true</span>
</span></span><span class="line"><span class="cl">  tags: pihole
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">- name: <span class="s2">&#34;Wait for server to come back&#34;</span>
</span></span><span class="line"><span class="cl">  local_action: wait_for <span class="nv">host</span><span class="o">={{</span> ansible_host <span class="o">}}</span> <span class="nv">port</span><span class="o">=</span><span class="m">22</span> <span class="nv">state</span><span class="o">=</span>started <span class="nv">delay</span><span class="o">=</span><span class="m">10</span>
</span></span><span class="line"><span class="cl">  become: <span class="nb">false</span>
</span></span></code></pre></div><p>But, as you can see, we referenced a <code>templates/setupVars.conf.j2</code> file. In our <code>ansible-homelab</code> directory, we’ll create a new directory called <code>templates</code> to house this file.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">mkdir templates/
</span></span><span class="line"><span class="cl">touch templates/setupVars.conf.j2
</span></span></code></pre></div><p>This file should contain your own configuration for PiHole. For this article, you can use the following basic configuration. The password encrypted here is <strong>pihole</strong>.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-ini" data-lang="ini"><span class="line"><span class="cl"><span class="na">QUERY_LOGGING</span><span class="o">=</span><span class="s">true</span>
</span></span><span class="line"><span class="cl"><span class="na">INSTALL_WEB</span><span class="o">=</span><span class="s">true</span>
</span></span><span class="line"><span class="cl"><span class="na">WEBPASSWORD</span><span class="o">=</span><span class="s">5536c470d038c11793b535e8c1176817c001d6f20a4704fa7908939be82e2922</span>
</span></span></code></pre></div><h3 id="docker-tasks">Docker tasks</h3>
<p>Now with our Docker host, we will need to install the Docker engine, setup a <code>docker-compose.yml</code> to configure our containers, then run docker-compose to start the containers.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">mkdir -p roles/docker/tasks/
</span></span><span class="line"><span class="cl">mkdir -p roles/docker/templates/docker_data
</span></span><span class="line"><span class="cl">touch roles/docker/tasks/main.yml
</span></span></code></pre></div><p>The contents of <code>main.yml</code> are as follows:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nn">---</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Add Apt signing key from official docker repo</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">apt_key</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">url</span><span class="p">:</span><span class="w"> </span><span class="l">https://download.docker.com/linux/debian/gpg</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">state</span><span class="p">:</span><span class="w"> </span><span class="l">present</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">add docker official repository for Debian Bullseye</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">apt_repository</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">repo</span><span class="p">:</span><span class="w"> </span><span class="l">deb [arch=amd64] https://download.docker.com/linux/debian bullseye stable</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">state</span><span class="p">:</span><span class="w"> </span><span class="l">present</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Index new repo into the cache</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">become</span><span class="p">:</span><span class="w"> </span><span class="kc">yes</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">apt</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;*&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">state</span><span class="p">:</span><span class="w"> </span><span class="l">latest</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">update_cache</span><span class="p">:</span><span class="w"> </span><span class="kc">yes</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">force_apt_get</span><span class="p">:</span><span class="w"> </span><span class="kc">yes</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">actually install docker</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">apt</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;docker-ce&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">state</span><span class="p">:</span><span class="w"> </span><span class="l">latest</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Install docker-compose</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">get_url</span><span class="p">:</span><span class="w"> 
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">url</span><span class="p">:</span><span class="w"> </span><span class="l">https://github.com/docker/compose/releases/download/v2.2.3/docker-compose-linux-x86_64</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">dest</span><span class="p">:</span><span class="w"> </span><span class="l">/usr/local/bin/docker-compose</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">mode</span><span class="p">:</span><span class="w"> </span><span class="s1">&#39;u+x,g+x&#39;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">install docker pip package</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">pip</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">name</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">docker</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">docker-compose</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">tags</span><span class="p">:</span><span class="w"> </span><span class="l">docker</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">create directory to house our docker-compose</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">file</span><span class="p">:</span><span class="w"> 
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/debian/docker/ </span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">state</span><span class="p">:</span><span class="w"> </span><span class="l">directory</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">owner</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">tags</span><span class="p">:</span><span class="w"> </span><span class="l">docker</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">copy docker-compose.yml</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">copy</span><span class="p">:</span><span class="w"> 
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">src</span><span class="p">:</span><span class="w"> </span><span class="l">templates/docker.conf.j2 </span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">dest</span><span class="p">:</span><span class="w"> </span><span class="l">/home/debian/docker/docker-compose.yml</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">owner</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">tags</span><span class="p">:</span><span class="w"> </span><span class="l">docker</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">create docker_data directory</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">file</span><span class="p">:</span><span class="w"> 
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l">/home/debian/docker/docker_data/ </span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">state</span><span class="p">:</span><span class="w"> </span><span class="l">directory</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">owner</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">tags</span><span class="p">:</span><span class="w"> </span><span class="l">docker</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">copy supporting files</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">copy</span><span class="p">:</span><span class="w"> 
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">src</span><span class="p">:</span><span class="w"> </span><span class="l">roles/docker/templates/docker_data/ </span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">dest</span><span class="p">:</span><span class="w"> </span><span class="l">/home/debian/docker/docker_data/</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">owner</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">group</span><span class="p">:</span><span class="w"> </span><span class="l">debian</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">tags</span><span class="p">:</span><span class="w"> </span><span class="l">docker</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Run docker-compose</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">docker_compose</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">project_src</span><span class="p">:</span><span class="w"> </span><span class="l">/home/debian/docker/</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">build</span><span class="p">:</span><span class="w"> </span><span class="kc">no</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">tags</span><span class="p">:</span><span class="w"> </span><span class="l">docker</span><span class="w">
</span></span></span></code></pre></div><p>The <code>docker_data</code> subdirectory can contain whatever files you want to bring over for your docker configuration. This could contain additional configuration files for containers you have defined in your <code>docker.conf.j2 template</code>.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-docker" data-lang="docker"><span class="line"><span class="cl">version: <span class="s2">&#34;3.7&#34;</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>services:<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>    portainer:<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>      container_name: portainer<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>      image: portainer/portainer-ce:latest<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>      restart: unless-stopped<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>      ports:<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>        - 8000:8000<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>        - 9000:9000<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>      volumes:<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>        - /var/run/docker.sock:/var/run/docker.sock<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span>        - /home/debian/docker/docker_data/:/data<span class="err">
</span></span></span></code></pre></div><h2 id="running-our-playbook">Running our playbook</h2>
<p>All of that hard work will pay off as soon as we execute the following command. Again, ignore any 10.0.5.x addresses in the screenshots or command output. For this article, that is equivalent to 192.168.1.x.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">ansible-playbook -i hosts --become -c debian playbook.yml
</span></span></code></pre></div><p>If you receive any errors, you can add <code>-vvv</code> to the previous command for verbose output.
<img loading="lazy" src="/posts/automating-your-homelab-p3-ansible/images/1*1RJ2DvbQj018E4flfBZESQ.webp" type="" alt=""  /></p>
<p>Once the command has completed, we’ll receive a play recap. This is an aggregated summary of the status of our tasks executed.</p>
<p><img loading="lazy" src="/posts/automating-your-homelab-p3-ansible/images/1*e4jxk6B6geMsnx9HiVYLRA.webp" type="" alt=""  /></p>
<p>If we navigate to <del><a href="http://192.168.1.210/admin">http://192.168.1.210/admin</a></del>, we will be presented with the pihole login page. Again, pihole was the encrypted password</p>
<p>Additionally, if we navigate to <del><a href="http://192.168.1.212:9000/">http://192.168.1.212:9000/</a></del>, we will be presented with Portainer’s initial password setup screen, from there we can access our containers inside Portainer.</p>
<p><img loading="lazy" src="/posts/automating-your-homelab-p3-ansible/images/1*XxDTr8dHrNCGf1cqXOC6IA.webp" type="" alt=""  /></p>
<h2 id="future-developments">Future developments</h2>
<ul>
<li>Make use of <a href="https://www.redhat.com/sysadmin/ansible-playbooks-secrets">Ansible Secrets</a>. While the pihole password was encrypted, its value is still stored in version control.</li>
<li>Explore more <a href="https://docs.ansible.com/ansible/latest/collections/index_module.html">modules</a> for specific uses. For example, the <a href="https://docs.ansible.com/ansible/latest/collections/community/general/ssh_config_module.html">ssh_config</a> and <a href="https://docs.ansible.com/ansible/latest/collections/ansible/builtin/user_module.html">user_module</a> modules.</li>
<li>Implement more containers under docker-compose. For example, implement a reverse proxy such as traefic to access all your services by subdomain.</li>
<li>Implement a CI/CD pipeline.</li>
<li>Look into <a href="https://www.ansible.com/products/controller">Ansible Tower</a></li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>In conclusion, I hope you have seen the value of using automation to build our homelab up with infrastructure as code. This three-part series hopefully has provided you with a solid foundation for building and managing your homelab environment.
With Ansible as a powerful tool in your arsenal, you can now automate repetitive tasks, ensure consistent configurations, and scale your homelab with ease. Embrace the possibilities and continue to enhance (or break) your homelab!</p>
]]></content:encoded></item><item><title>Automating Your Homelab with Proxmox, Cloud-init, Terraform, and Ansible — Part 2- Deploying your VMs with Terraform</title><link>https://fz42.net/posts/automating-your-homelab-p2-terraform/</link><pubDate>Mon, 03 Apr 2023 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/automating-your-homelab-p2-terraform/</guid><description>&lt;p&gt;Picking back up where we left off in &lt;a href="https://fz42.net/posts/automating-your-homelab-p1-cloudinit/"&gt;Part 1&lt;/a&gt; of our homelab automation series, we now have a cloud-init image that we can use to quickly and easily configure new virtual machines in Proxmox. However, manually creating and configuring each VM can still be a tedious and time-consuming task, especially if you have a lot of them to manage. This is where Terraform comes in.&lt;/p&gt;
&lt;h2 id="overview-of-terraform"&gt;Overview of Terraform&lt;/h2&gt;
&lt;p&gt;Terraform is an open-source infrastructure as code tool that allows you to define and manage your infrastructure as code. This code is a declarative language that is simple to write. In this article, we will explore how to use Terraform to automate the creation of VMs in Proxmox, using the previously generated cloud-init image. But before we can do that, we need to set up Proxmox to allow Terraform to authenticate and interact with the API.&lt;/p&gt;</description><content:encoded><![CDATA[<p>Picking back up where we left off in <a href="https://fz42.net/posts/automating-your-homelab-p1-cloudinit/">Part 1</a> of our homelab automation series, we now have a cloud-init image that we can use to quickly and easily configure new virtual machines in Proxmox. However, manually creating and configuring each VM can still be a tedious and time-consuming task, especially if you have a lot of them to manage. This is where Terraform comes in.</p>
<h2 id="overview-of-terraform">Overview of Terraform</h2>
<p>Terraform is an open-source infrastructure as code tool that allows you to define and manage your infrastructure as code. This code is a declarative language that is simple to write. In this article, we will explore how to use Terraform to automate the creation of VMs in Proxmox, using the previously generated cloud-init image. But before we can do that, we need to set up Proxmox to allow Terraform to authenticate and interact with the API.</p>
<h2 id="configuring-api-users-permissions-and-tokens-in-proxmox">Configuring API Users, Permissions, and Tokens in Proxmox</h2>
<p>After logging into Proxmox, under the <strong>Datacenter</strong> menu, click on Users <strong>under</strong> the <strong>Permissions</strong> menu. Here, we will click the Add button to configure a new user. Create a PAM standard user with a username and ensure that it’s enabled.</p>
<p><img loading="lazy" src="/posts/automating-your-homelab-p2-terraform/images/1*ag1E_NnLdSOQS98Zr-tyrw.webp" type="" alt=""  /></p>
<p>Now, under the same <strong>Permissions</strong> menu, we want to click on <strong>API Tokens</strong>. Create a new API Token by selecting the user we just created, ensure that <strong>Privilege Separation</strong> is enabled, and name the <strong>Token ID</strong>: <code>terraform_token_id</code>. As you’ll be warned, make sure you copy down the Token Secret, as it will only be displayed once.</p>
<p><img loading="lazy" src="/posts/automating-your-homelab-p2-terraform/images/1*CVe1Fj8ouw6P-quZ5IRQew.webp" type="" alt=""  />
Now go back up to the <strong>Permissions</strong> menu, and we’ll add new user permissions by clicking the <strong>User Permission</strong> menu item from the Add button dropdown.</p>
<p><img loading="lazy" src="/posts/automating-your-homelab-p2-terraform/images/1*sLxkF7IP41THJ3kYeujYaw.webp" type="" alt=""  /></p>
<p>We’re going to add three, one for the path of <code>/</code>, <code>/storage/local</code>, and <code>/storage/local-lvm</code>. Your paths may vary depending on how your storage is configured. The roles and paths are below.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">/                    terraform@pam   PVEVMAdmin
</span></span><span class="line"><span class="cl">/storage/local       terraform@pam   Administrator
</span></span><span class="line"><span class="cl">/storage/local-lvm   terraform@pam   Administrator
</span></span></code></pre></div><p><img loading="lazy" src="/posts/automating-your-homelab-p2-terraform/images/1*UQMBtj_6QW2bMYccDLJoPw.webp" type="" alt=""  />
Now, Terraform will have access to Proxmox’s API to create the VMs based on our configuration file.</p>
<p>The last thing we need to do on Proxmox’s end is install terraform. It’s not in the official repos for Debian, so we’ll add a new repository and install it onto Proxmox.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl"><span class="c1"># Install terraform</span>
</span></span><span class="line"><span class="cl">apt install lsb-release
</span></span><span class="line"><span class="cl">curl -fsSL https://apt.releases.hashicorp.com/gpg <span class="p">|</span> apt-key add -
</span></span><span class="line"><span class="cl"><span class="nb">echo</span> <span class="s2">&#34;deb [arch=</span><span class="k">$(</span>dpkg --print-architecture<span class="k">)</span><span class="s2">] https://apt.releases.hashicorp.com </span><span class="k">$(</span>lsb_release -cs<span class="k">)</span><span class="s2"> main&#34;</span> &gt;&gt; /etc/apt/sources.list.d/terraform.list
</span></span><span class="line"><span class="cl">apt update <span class="o">&amp;&amp;</span> apt install terraform
</span></span></code></pre></div><h2 id="creating-a-terraform-configuration">Creating a Terraform Configuration</h2>
<p>For brevity, we’ll only be configuring two virtual machines, but the process is repeatable based on however many VMs you want to host.</p>
<p>If you don’t want to work through these steps manually, you can clone the repository located at <del><a href="https://github.com/fatzombi/terraform-homelab">https://github.com/fatzombi/terraform-homelab</a></del>.</p>
<p>We’ll be creating the following directory structure and files:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">~/terraform-homelab
</span></span><span class="line"><span class="cl">    - main.tf
</span></span><span class="line"><span class="cl">    - vars.tf
</span></span><span class="line"><span class="cl">    - pihole.tf
</span></span><span class="line"><span class="cl">    - homebridge.tf
</span></span></code></pre></div><h3 id="maintf">main.tf</h3>
<p>This file contains a provider, in this case we’re using the <code>telmate/proxmox</code> provider to interact with the Proxmox API. You won’t have to change anything within this file unless you wish to enable TLS verification.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-terraform" data-lang="terraform"><span class="line"><span class="cl"><span class="nx">terraform</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nx">required_providers</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">proxmox</span> <span class="o">=</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">      <span class="nx">source</span> <span class="o">=</span> <span class="s2">&#34;telmate/proxmox&#34;</span>
</span></span><span class="line"><span class="cl">      <span class="nx">version</span> <span class="o">=</span> <span class="s2">&#34;2.7.4&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kr">provider</span> <span class="s2">&#34;proxmox&#34;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nx">pm_api_url</span> <span class="o">=</span> <span class="nx">vars</span><span class="p">.</span><span class="nx">api_url</span>
</span></span><span class="line"><span class="cl">  <span class="nx">pm_api_token_id</span> <span class="o">=</span> <span class="nb">var</span><span class="p">.</span><span class="nx">token_id</span>
</span></span><span class="line"><span class="cl">  <span class="nx">pm_api_token_secret</span> <span class="o">=</span> <span class="nb">var</span><span class="p">.</span><span class="nx">token_secret</span>
</span></span><span class="line"><span class="cl">  <span class="nx">pm_tls_insecure</span> <span class="o">=</span> <span class="kc">true</span><span class="c1"> # Change to false if you have your 
</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">}</span>
</span></span></code></pre></div><h3 id="varstf">vars.tf</h3>
<p>This file contains all the variables that are used in <code>main.tf</code> and the respective .tf files for each VM we define. The last variables are for configuring static IP addresses on our VMs, but this step is optional.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-terraform" data-lang="terraform"><span class="line"><span class="cl"><span class="kr">variable</span> <span class="s2">&#34;ssh_key&#34;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">default</span> <span class="o">=</span> <span class="s2">&#34;ssh-rsa .....&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="kr">variable</span> <span class="s2">&#34;api_url&#34;</span> <span class="p">{</span><span class="c1">
</span></span></span><span class="line"><span class="cl"><span class="c1">    # The Proxmox Web UI address, with /api2/json added to it.
</span></span></span><span class="line"><span class="cl"><span class="c1"></span>    <span class="nx">default</span> <span class="o">=</span> <span class="s2">&#34;https://192.168.1.15:8006/api2/json&#34;</span> 
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="kr">variable</span> <span class="s2">&#34;proxmox_host&#34;</span> <span class="p">{</span><span class="c1">
</span></span></span><span class="line"><span class="cl"><span class="c1">    # The name of the Proxmox server listed under Datacenter
</span></span></span><span class="line"><span class="cl"><span class="c1"></span>    <span class="nx">default</span> <span class="o">=</span> <span class="s2">&#34;...&#34;</span> 
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="kr">variable</span> <span class="s2">&#34;template_name&#34;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nx">default</span> <span class="o">=</span> <span class="s2">&#34;debian-11-cloudinit-template&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="kr">variable</span> <span class="s2">&#34;token_id&#34;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nx">default</span> <span class="o">=</span> <span class="s2">&#34;terraform@pam!terraform_token_id&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="kr">variable</span> <span class="s2">&#34;token_secret&#34;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nx">default</span> <span class="o">=</span> <span class="s2">&#34;...&#34;</span><span class="c1"> # Enter your API Secret here
</span></span></span><span class="line"><span class="cl"><span class="c1"></span><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="kr">variable</span> <span class="s2">&#34;ipconfig_pihole&#34;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nx">default</span> <span class="o">=</span> <span class="s2">&#34;ip=192.168.1.16/24,gw=192.168.1.1&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="kr">variable</span> <span class="s2">&#34;ipconfig_homebridge&#34;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nx">default</span> <span class="o">=</span> <span class="s2">&#34;ip=192.168.1.17/24,gw=192.168.1.1&#34;</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><h3 id="piholetf">pihole.tf</h3>
<p>Now we can define the resources allocated to our pihole vm. The primary areas of interest will be <strong>cores</strong>, <strong>sockets</strong>, memory, <strong>and</strong> <strong>disk.size</strong>.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-terraform" data-lang="terraform"><span class="line"><span class="cl"><span class="kr">resource</span> <span class="s2">&#34;proxmox_vm_qemu&#34;</span> <span class="s2">&#34;pihole&#34;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nb">count</span> <span class="o">=</span> <span class="m">1</span>
</span></span><span class="line"><span class="cl">  <span class="nx">name</span> <span class="o">=</span> <span class="s2">&#34;pihole&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="nx">target_node</span> <span class="o">=</span> <span class="nb">var</span><span class="p">.</span><span class="nx">proxmox_host</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">  <span class="nx">clone</span> <span class="o">=</span> <span class="nb">var</span><span class="p">.</span><span class="nx">template_name</span>
</span></span><span class="line"><span class="cl">  <span class="nx">agent</span> <span class="o">=</span> <span class="m">1</span>
</span></span><span class="line"><span class="cl">  <span class="nx">os_type</span> <span class="o">=</span> <span class="s2">&#34;cloud-init&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="nx">cores</span> <span class="o">=</span> <span class="m">1</span>
</span></span><span class="line"><span class="cl">  <span class="nx">sockets</span> <span class="o">=</span> <span class="m">1</span>
</span></span><span class="line"><span class="cl">  <span class="nx">cpu</span> <span class="o">=</span> <span class="s2">&#34;host&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="nx">memory</span> <span class="o">=</span> <span class="m">1024</span>
</span></span><span class="line"><span class="cl">  <span class="nx">scsihw</span> <span class="o">=</span> <span class="s2">&#34;virtio-scsi-pci&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="nx">bootdisk</span> <span class="o">=</span> <span class="s2">&#34;scsi0&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="nx">disk</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">slot</span> <span class="o">=</span> <span class="m">0</span>
</span></span><span class="line"><span class="cl">    <span class="nx">size</span> <span class="o">=</span> <span class="s2">&#34;8G&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="nx">type</span> <span class="o">=</span> <span class="s2">&#34;scsi&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="nx">storage</span> <span class="o">=</span> <span class="s2">&#34;local-lvm&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="nx">iothread</span> <span class="o">=</span> <span class="m">1</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="nx">network</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">model</span> <span class="o">=</span> <span class="s2">&#34;virtio&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="nx">bridge</span> <span class="o">=</span> <span class="s2">&#34;vmbr0&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="nx">lifecycle</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">ignore_changes</span> <span class="o">=</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">      <span class="nx">network</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">]</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="nx">ipconfig0</span> <span class="o">=</span> <span class="nb">var</span><span class="p">.</span><span class="nx">ipconfig_pihole</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="nx">sshkeys</span> <span class="o">=</span> <span class="o">&lt;&lt;EOF</span><span class="s">
</span></span></span><span class="line"><span class="cl"><span class="s">  ${var.ssh_key}
</span></span></span><span class="line"><span class="cl"><span class="s">  </span><span class="o">EOF</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><h3 id="homebridgetf">homebridge.tf</h3>
<p>This file will be similar to our <code>pihole.tf</code>; however, there’s an additional section where we pass through a USB device from the host to the VM.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-terraform" data-lang="terraform"><span class="line"><span class="cl"><span class="kr">resource</span> <span class="s2">&#34;proxmox_vm_qemu&#34;</span> <span class="s2">&#34;homebridge&#34;</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">  <span class="nb">count</span> <span class="o">=</span> <span class="m">1</span>
</span></span><span class="line"><span class="cl">  <span class="nx">name</span> <span class="o">=</span> <span class="s2">&#34;homebridge&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="nx">target_node</span> <span class="o">=</span> <span class="nb">var</span><span class="p">.</span><span class="nx">proxmox_host</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">  <span class="nx">clone</span> <span class="o">=</span> <span class="nb">var</span><span class="p">.</span><span class="nx">template_name</span>
</span></span><span class="line"><span class="cl">  <span class="nx">agent</span> <span class="o">=</span> <span class="m">1</span>
</span></span><span class="line"><span class="cl">  <span class="nx">os_type</span> <span class="o">=</span> <span class="s2">&#34;cloud-init&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="nx">cores</span> <span class="o">=</span> <span class="m">2</span>
</span></span><span class="line"><span class="cl">  <span class="nx">sockets</span> <span class="o">=</span> <span class="m">1</span>
</span></span><span class="line"><span class="cl">  <span class="nx">cpu</span> <span class="o">=</span> <span class="s2">&#34;host&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="nx">memory</span> <span class="o">=</span> <span class="m">8192</span>
</span></span><span class="line"><span class="cl">  <span class="nx">scsihw</span> <span class="o">=</span> <span class="s2">&#34;virtio-scsi-pci&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="nx">bootdisk</span> <span class="o">=</span> <span class="s2">&#34;scsi0&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="nx">disk</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">slot</span> <span class="o">=</span> <span class="m">0</span>
</span></span><span class="line"><span class="cl">    <span class="nx">size</span> <span class="o">=</span> <span class="s2">&#34;50G&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="nx">type</span> <span class="o">=</span> <span class="s2">&#34;scsi&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="nx">storage</span> <span class="o">=</span> <span class="s2">&#34;local-lvm&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="nx">iothread</span> <span class="o">=</span> <span class="m">1</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="nx">network</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">model</span> <span class="o">=</span> <span class="s2">&#34;virtio&#34;</span>
</span></span><span class="line"><span class="cl">    <span class="nx">bridge</span> <span class="o">=</span> <span class="s2">&#34;vmbr0&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="nx">lifecycle</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">ignore_changes</span> <span class="o">=</span> <span class="p">[</span>
</span></span><span class="line"><span class="cl">      <span class="nx">network</span><span class="p">,</span>
</span></span><span class="line"><span class="cl">    <span class="p">]</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="nx">serial</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="nx">id</span> <span class="o">=</span> <span class="m">0</span>
</span></span><span class="line"><span class="cl">    <span class="nx">type</span> <span class="o">=</span> <span class="s2">&#34;/dev/tty0&#34;</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="nx">ipconfig0</span> <span class="o">=</span> <span class="nb">var</span><span class="p">.</span><span class="nx">ipconfig_homebridge</span>
</span></span><span class="line"><span class="cl">  
</span></span><span class="line"><span class="cl">  <span class="nx">sshkeys</span> <span class="o">=</span> <span class="o">&lt;&lt;EOF</span><span class="s">
</span></span></span><span class="line"><span class="cl"><span class="s">  ${var.ssh_key}
</span></span></span><span class="line"><span class="cl"><span class="s">  </span><span class="o">EOF</span>
</span></span><span class="line"><span class="cl"><span class="p">}</span>
</span></span></code></pre></div><h2 id="deploying-vms-with-terraform">Deploying VMs with Terraform</h2>
<p>Now that we have our Terraform configuration, there are only a few more steps we need to take.</p>
<ul>
<li><code>terraform init</code></li>
<li><code>terraform plan</code></li>
<li><code>terraform apply</code></li>
</ul>
<p>⠀If everything was successful, you should be able to see your newly created virtual machines in Proxmox. For example, here are the details of our <code>pihole</code> VM.
<img loading="lazy" src="/posts/automating-your-homelab-p2-terraform/images/1*yIquvbYTvZE6b6QhC9k8Fg.webp" type="" alt=""  /></p>
<h2 id="conclusion">Conclusion</h2>
<p>In this article, we have covered how to configure Proxmox to enable Terraform to automatically create virtual machines based on our previously generated cloud-init image. We started by creating a new user and generating an API token with the appropriate permissions to allow Terraform to interact with our Proxmox instance. We then installed and configured the Proxmox Terraform provider, which allowed us to define our virtual machines as infrastructure-as-code using Terraform. We created a basic Terraform configuration that defined our VMs, its resources, and attached our cloud-init image to it. Finally, we ran the terraform apply command, which created our new virtual machines with the defined specifications.</p>
<p>By using Terraform, we can automate the process of creating virtual machines and ensure consistent configurations across all of our instances. With the Proxmox Terraform provider, we can easily manage our infrastructure as code and make changes in a repeatable, predictable way. In the next article, we will cover how to use Ansible to perform software-related tasks such as installing dependencies for a particular role the VM will offer.</p>
]]></content:encoded></item><item><title>Automating Your Homelab with Proxmox, Cloud-init, Terraform, and Ansible — Part 1- Configuring a base image with Cloud-Init</title><link>https://fz42.net/posts/automating-your-homelab-p1-cloudinit/</link><pubDate>Sat, 11 Mar 2023 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/automating-your-homelab-p1-cloudinit/</guid><description>&lt;p&gt;Cloud-init is an open-source package which allows the automation of the initial setup and configuration of virtual machines, making it much easier to manage and deploy them. With Cloud-init, we can define a set of instructions, contained within a user-data file, which tells each virtual machine what to do when it starts up. This user-data file can contain instructions such as configuring the network interfaces, setting up user accounts, or installing software packages. By using Cloud-init, we can streamline the process of setting up virtual machines and ensure consistent configurations across all the instances.&lt;/p&gt;</description><content:encoded><![CDATA[<p>Cloud-init is an open-source package which allows the automation of the initial setup and configuration of virtual machines, making it much easier to manage and deploy them. With Cloud-init, we can define a set of instructions, contained within a user-data file, which tells each virtual machine what to do when it starts up. This user-data file can contain instructions such as configuring the network interfaces, setting up user accounts, or installing software packages. By using Cloud-init, we can streamline the process of setting up virtual machines and ensure consistent configurations across all the instances.</p>
<p>For this process, we will perform these actions from our Proxmox instance.</p>
<h2 id="creating-a-base-image-with-cloud-init">Creating a base image with Cloud-init</h2>
<p>First, we need to download and extract a cloud-init image, in my case, I want to use Debian as my base.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl"><span class="nb">cd</span> /var/lib/vz/template/iso/
</span></span><span class="line"><span class="cl">wget https://cloud.debian.org/images/cloud/bullseye/latest/debian-11-generic-amd64.tar.xz 
</span></span><span class="line"><span class="cl">tar -xf debian-11-generic-amd64.tar.xz
</span></span></code></pre></div><p>Let’s go ahead and rename the raw disk, and delete the original archive we downloaded.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">mv disk.raw debian-11-generic-amd64.img
</span></span><span class="line"><span class="cl">rm debian-11-generic-amd64.tar.xz
</span></span></code></pre></div><p>There are a few dependencies that we’ll need installed in our Proxmox instance.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl"><span class="nb">echo</span> <span class="s2">&#34;deb http://download.proxmox.com/debian bullseye pve-no-subscription&#34;</span> &gt;&gt; /etc/apt/sources.list.d/pve-enterprise.list
</span></span><span class="line"><span class="cl">apt update -y <span class="o">&amp;&amp;</span> apt install libguestfs-tools -y
</span></span></code></pre></div><p><img loading="lazy" src="/posts/automating-your-homelab-p1-cloudinit/images/1*cXBwBsDI1e44BAYkD7zyqA.png.webp" type="" alt=""  /></p>
<p>At this time, you could perform additional actions against the VM. See <del><a href="https://www.libguestfs.org/virt-customize.1.html">virt-customize</a></del> for more options. However, in our case, we will handle additional configurations with Ansible.</p>
<p>Now we can create a template from this Cloud-init disk.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">qm create <span class="m">9001</span> --name <span class="s2">&#34;debian-11-cloudinit-template&#34;</span> --memory <span class="m">2048</span> --cores <span class="m">2</span> --net0 virtio,bridge<span class="o">=</span>vmbr0
</span></span><span class="line"><span class="cl">qm importdisk <span class="m">9001</span> debian-11-generic-amd64.img local-lvm
</span></span><span class="line"><span class="cl">qm <span class="nb">set</span> <span class="m">9001</span> --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-9001-disk-0
</span></span><span class="line"><span class="cl">qm <span class="nb">set</span> <span class="m">9001</span> --boot c --bootdisk scsi0
</span></span><span class="line"><span class="cl">qm <span class="nb">set</span> <span class="m">9001</span> --ide2 local-lvm:cloudinit
</span></span><span class="line"><span class="cl">qm <span class="nb">set</span> <span class="m">9001</span> --serial0 socket --vga serial0
</span></span><span class="line"><span class="cl">qm <span class="nb">set</span> <span class="m">9001</span> –-agent <span class="m">1</span>
</span></span><span class="line"><span class="cl">qm template <span class="m">9001</span>
</span></span></code></pre></div><h2 id="manually-using-the-base-image-to-create-a-virtual-machine">Manually using the base image to create a Virtual Machine</h2>
<p>At this point, we can click on <strong>More -&gt; Clone</strong> under this template in Proxmox.</p>
<p><img loading="lazy" src="/posts/automating-your-homelab-p1-cloudinit/images/1*Zebvbl-o2dX9ku90yylaXg.webp" type="" alt=""  /></p>
<p><img loading="lazy" src="/posts/automating-your-homelab-p1-cloudinit/images/1*oJB6gJKHRw1YtR44yhnzdQ.webp" type="" alt=""  /></p>
<p>If all is successful, the newly created VM will appear in our Virtual Machine list and it should boot successfully.</p>
<p><img loading="lazy" src="/posts/automating-your-homelab-p1-cloudinit/images/1*5cWtEBv9hhG5ll3Bw0VClQ.webp" type="" alt=""  /></p>
<h2 id="conclusion">Conclusion</h2>
<p>In conclusion, configuring a base image with Cloud-init is a crucial step in automating your homelab using Proxmox, Terraform, and Ansible. With Cloud-init, we can automate the initial setup and configuration of virtual machines, ensuring consistent configurations across all instances. By following the steps outlined in this article, you can easily create a Cloud-init base image from a Debian distribution and customize it to suit your needs. Additionally, you can create a template from this Cloud-init disk and use it to create multiple virtual machines. With this base image, you can save time and effort when deploying new virtual machines. In the next article, we will cover how to deploy virtual machines using Terraform.</p>
]]></content:encoded></item><item><title>Automating Your Homelab with Proxmox, Cloud-init, Terraform, and Ansible — Introduction</title><link>https://fz42.net/posts/automating-your-homelab-p0-intro/</link><pubDate>Fri, 10 Mar 2023 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/automating-your-homelab-p0-intro/</guid><description>Introduction article to building a homelab with Proxmox, Cloud-init, Terraform, and Ansible</description><content:encoded><![CDATA[<p>Welcome to our blog series on building a homelab with Proxmox, Cloud-init, Terraform, and Ansible! In this series, I’ll guide you through the process of setting up a fully automated homelab using open-source tools that are easy o use and are highly customizable.</p>
<p>Before we dive into the details, it’s assumed that you are familiar what a homelab is, if you don’t, I recommend you checkout the following resources:</p>
<ul>
<li><del><a href="https://www.reddit.com/r/homelab/comments/5gz4yp/stumbled_into_rhomelab_start_here/">Stumbled into /r/homelab? Start Here!</a></del></li>
<li><del><a href="https://www.reddit.com/r/homelab/wiki/introduction">An Introduction to Homelabs</a></del></li>
</ul>
<p>Additionally, it’s important to note that this series won’t cover the installation and configuration of Proxmox itself. Proxmox is a powerful and versatile hypervisor that allows you to run multiple virtual machines or containers on a single physical host. It’s widely used in both home and enterprise settings (not as much as VMWare), and offers a wide range of features and capabilities. However, setting up Proxmox can be a complex process that requires a solid understanding of virtualization, networking, and system administration.</p>
<p>Assuming you have Proxmox up and running, the rest of this series will focus on using Cloud-init, Terraform, and Ansible to create a base image/template, deploy virtual machines, and perform software-related tasks such as installing dependencies and configuring software to run on each virtual machine. By automating these tasks, you’ll be able to quickly and easily set up a homelab that meets your specific needs, without having to manually configure each virtual machine. An added benefit is we can easily recreate the environment if something were to break. So, let’s get started!</p>
<p><a href="https://fz42.net/posts/automating-your-homelab-p1-cloudinit/">Part 1: Configuring a base image with Cloud-Init</a></p>
<p><a href="https://fz42.net/posts/automating-your-homelab-p2-terraform/">Part 2: Deploying VMs with Terraform</a></p>
<p><a href="https://fz42.net/posts/automating-your-homelab-p3-ansible/">Part 3: Automating with Ansible</a></p>
]]></content:encoded></item><item><title>Deploying a Jekyll website to AWS S3 with GitHub Actions and AWS CloudFormation</title><link>https://fz42.net/posts/jekyll-s3-githubactions/</link><pubDate>Sun, 01 Jan 2023 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/jekyll-s3-githubactions/</guid><description>&lt;p&gt;In this tutorial, we will configure a static website using Jekyll, GitHub Actions, AWS S3, AWS Route 53, AWS Certificate Manager, AWS CloudFront, and AWS CloudFormation. And yes, that sounds like a mouthful, but trust me, it’s not as intimidating as it sounds.&lt;/p&gt;
&lt;p&gt;To begin, we’ll highlight the main technologies being used and the role they play in our solution.&lt;/p&gt;
&lt;p&gt;Next, I’ll dive into the roles of each AWS offering used in this architecture, including how CloudFormation is used to create and manage the infrastructure for our website, how Route 53 and Certificate Manager are used to manage our domain, DNS records, and TLS certificates, and how CloudFront will be used for content delivery and caching.&lt;/p&gt;</description><content:encoded><![CDATA[<p>In this tutorial, we will configure a static website using Jekyll, GitHub Actions, AWS S3, AWS Route 53, AWS Certificate Manager, AWS CloudFront, and AWS CloudFormation. And yes, that sounds like a mouthful, but trust me, it’s not as intimidating as it sounds.</p>
<p>To begin, we’ll highlight the main technologies being used and the role they play in our solution.</p>
<p>Next, I’ll dive into the roles of each AWS offering used in this architecture, including how CloudFormation is used to create and manage the infrastructure for our website, how Route 53 and Certificate Manager are used to manage our domain, DNS records, and TLS certificates, and how CloudFront will be used for content delivery and caching.</p>
<p>Lastly, I’ll discuss how GitHub Actions are used to trigger the workflows in AWS that deploy updates to our website.</p>
<h1 id="solution-overview">Solution Overview</h1>
<p><img loading="lazy" src="/posts/jekyll-s3-githubactions/images/1*9I6tLbfSvyB8qoS6sdecPA.webp" type="" alt=""  /></p>
<h1 id="setting-up-aws-cloudformation">Setting up AWS CloudFormation</h1>
<p>AWS CloudFormation is a powerful service that allows you to create and manage your infrastructure as code. Using CloudFormation, we can define our infrastructure using templates written in either JSON or YAML, then we can create and update those resources using <code>awscli</code> or AWS Management Console. If you are familiar with <code>docker-compose</code>, this may be familiar to you.</p>
<p>In our solution, we are using AWS S3, Route 53, CloudFront, and Certificate Manager. These are all the resources that we will define in our YAML template, which our stack will be composed of.</p>
<h2 id="cloudformation-stack-template">CloudFormation Stack Template</h2>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="c"># website-stack.yml </span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">Parameters</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">DomainName</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Description</span><span class="p">:</span><span class="w"> </span><span class="l">The domain of the website</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Type</span><span class="p">:</span><span class="w"> </span><span class="l">String</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">Resources</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">HostedZone</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Type</span><span class="p">:</span><span class="w"> </span><span class="l">AWS::Route53::HostedZone</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Properties</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">Name</span><span class="p">:</span><span class="w"> </span>!<span class="l">Ref DomainName</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">HostedZoneConfig</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">Comment</span><span class="p">:</span><span class="w"> </span><span class="l">Managed by CloudFormation</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"> 
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">WebsiteBucket</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Type</span><span class="p">:</span><span class="w"> </span><span class="l">AWS::S3::Bucket</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Properties</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">BucketName</span><span class="p">:</span><span class="w"> </span>!<span class="l">Ref DomainName</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">WebsiteConfiguration</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">IndexDocument</span><span class="p">:</span><span class="w"> </span><span class="l">index.html</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">ErrorDocument</span><span class="p">:</span><span class="w"> </span><span class="l">error.html</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">Certificate</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Type</span><span class="p">:</span><span class="w"> </span><span class="l">AWS::CertificateManager::Certificate</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">DependsOn</span><span class="p">:</span><span class="w"> </span><span class="l">HostedZone</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Properties</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">DomainName</span><span class="p">:</span><span class="w"> </span>!<span class="l">Ref DomainName</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">SubjectAlternativeNames</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- !<span class="l">Sub &#39;www.${DomainName}&#39;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">ValidationMethod</span><span class="p">:</span><span class="w"> </span><span class="l">DNS</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">DomainValidationOptions</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>- <span class="nt">DomainName</span><span class="p">:</span><span class="w"> </span>!<span class="l">Ref DomainName</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">HostedZoneId</span><span class="p">:</span><span class="w"> </span>!<span class="l">Ref HostedZone</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>- <span class="nt">DomainName</span><span class="p">:</span><span class="w"> </span>!<span class="l">Sub &#39;www.${DomainName}&#39;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">HostedZoneId</span><span class="p">:</span><span class="w"> </span>!<span class="l">Ref HostedZone</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">Distribution</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Type</span><span class="p">:</span><span class="w"> </span><span class="l">AWS::CloudFront::Distribution</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Properties</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">DistributionConfig</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">Aliases</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>- !<span class="l">Ref DomainName</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>- <span class="l">www.!Ref DomainName</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">Origins</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>- <span class="nt">DomainName</span><span class="p">:</span><span class="w"> </span>!<span class="l">Sub &#39;${WebsiteBucket}.s3.amazonaws.com&#39;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">Id</span><span class="p">:</span><span class="w"> </span><span class="l">S3Origin</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">S3OriginConfig</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span><span class="nt">OriginAccessIdentity</span><span class="p">:</span><span class="w"> </span>!<span class="l">Sub &#39;origin-access-identity/cloudfront/${WebsiteBucket}&#39;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">DefaultCacheBehavior</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">TargetOriginId</span><span class="p">:</span><span class="w"> </span><span class="l">S3Origin</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">ViewerProtocolPolicy</span><span class="p">:</span><span class="w"> </span><span class="l">redirect-to-https</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">Compress</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">ForwardedValues</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">            </span><span class="nt">QueryString</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">ViewerCertificate</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">AcmCertificateArn</span><span class="p">:</span><span class="w"> </span>!<span class="l">Ref Certificate</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">SslSupportMethod</span><span class="p">:</span><span class="w"> </span><span class="l">sni-only</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">RecordSetGroup</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Type</span><span class="p">:</span><span class="w"> </span><span class="l">AWS::Route53::RecordSetGroup</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">Properties</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">HostedZoneId</span><span class="p">:</span><span class="w"> </span>!<span class="l">Ref HostedZone</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">RecordSets</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">Name</span><span class="p">:</span><span class="w"> </span>!<span class="l">Ref DomainName</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">Type</span><span class="p">:</span><span class="w"> </span><span class="l">A</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">AliasTarget</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">HostedZoneId</span><span class="p">:</span><span class="w"> </span><span class="l">Z2FDTNDATAQYW2</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">          </span><span class="nt">DNSName</span><span class="p">:</span><span class="w"> </span>!<span class="l">GetAtt Distribution.DomainName</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="nt">Name</span><span class="p">:</span><span class="w"> </span>!<span class="l">Sub &#39;www.${DomainName}&#39;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">Type</span><span class="p">:</span><span class="w"> </span><span class="l">CNAME</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">ResourceRecords</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span>- !<span class="l">GetAtt Distribution.DomainName</span><span class="w">
</span></span></span></code></pre></div><p>The CloudFormation template creates the five resources we mentioned above:</p>
<ol>
<li>A Route53 Hosted Zone, which will be used to store the DNS records for our domain.</li>
<li>A S3 bucket called <code>example.com</code> that will contain the files for our Jekyll site. We are configuring the bucket as a static website, with an IndexDocument of <code>index.html</code> and an ErrorDocument of <code>error.html</code>.</li>
<li>A Certificate Manager certificate that will be used to secure our website with TLS. This certificate will be valid whether a user enters example.com or <a href="https://www.example.com">www.example.com</a> to access our website.</li>
<li>A CloudFront distribution that points to our S3 bucket and supports both <code>example.com</code> and <code>www.example.com</code>.</li>
<li>A RecordSetGroup which creates the DNS A and CNAME records required to map our domain to the CloudFront distribution, which in turn points to our S3 bucket which contains our static website.</li>
</ol>
<p>Now that we have the contents of our CloudFormation stack, save it to your file system. Go over to AWS Management Console and search for the CloudFormation product.</p>
<p><img loading="lazy" src="/posts/jekyll-s3-githubactions/images/1*5rNVE5LetmM4PRBFh0DPug.webp" type="" alt=""  /></p>
<h1 id="deploying-the-stack">Deploying the stack</h1>
<p>Click on <strong>Create stack</strong> and upload the YAML file as our template.</p>
<p><img loading="lazy" src="/posts/jekyll-s3-githubactions/images/1*vCzYWHNTKEuiLJCoonhn5A.webp" type="" alt=""  /></p>
<p>Click on <strong>Next</strong>, name our stack however you’d like and enter your domain name. Then click <strong>Next</strong>, <strong>Next</strong>, then <strong>Submit</strong>.</p>
<p>Now we can monitor the <strong>Events</strong> tab to check the progress of the resources CloudFormation is creating. The majority of the resources should be created within 5 minutes. However, the certificate will likely remain in <strong>CREATE_IN_PROGRESS</strong> status for up to an hour. At this time, you can visit each of the resources in AWS Management Console. If we navigate to the Certificate Manager, we will likely see that our domains are <strong>Pending validation</strong>.</p>
<p><img loading="lazy" src="/posts/jekyll-s3-githubactions/images/1*KloMJYRTaj4YrRrpp1u_sw.webp" type="" alt=""  /></p>
<p>After a couple of minutes, we should see the CNAME records created under our Route 53 Hosted Zone.</p>
<p><img loading="lazy" src="/posts/jekyll-s3-githubactions/images/1*I3Il8X2_fkgiGyLQAe83FQ.webp" type="" alt=""  /></p>
<p>Unfortunately, Certificate Manager can take a few hours to validate our domain.</p>
<p>At this point, we can continue with the tutorial. If your DNS servers haven’t updated, you can navigate to your S3 bucket in AWS Management Console, click the <strong>Properties</strong> tab, scroll to the bottom and copy the <strong>Bucket website endpoint</strong> URL under the *<em>Static website hosting</em> section. If you navigate to that URL, you’ll receive a 403 Forbidden response. This is expected, as we haven’t deployed our Jekyll files to the bucket yet.</p>
<p>To test, you can upload a file to your S3 bucket. You’ll need to select the file and click on <strong>Make public using ACL</strong>, under the <strong>Actions</strong> dropdown. Then click on the <strong>Copy URL</strong> button and navigate to that page, you should see the contents of your file.</p>
<h1 id="setting-up-github-secrets-and-actions">Setting up GitHub Secrets and Actions</h1>
<p>In this tutorial, I am assuming you have an IAM user created that will be used for GitHub Actions. In that case, you’ll need to have your AWS Access Key ID and Secret Access Key values. If you don’t, you’ll want to check out one of the following articles to obtain this:</p>
<ul>
<li><a href="https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html">https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html</a></li>
<li><a href="https://vasko.io/jekyll/aws/2017/08/03/jekyll-s3-cloudfront.html">https://vasko.io/jekyll/aws/2017/08/03/jekyll-s3-cloudfront.html</a></li>
<li><a href="https://betterprogramming.pub/build-a-static-website-with-jekyll-and-automatically-deploy-it-to-aws-s3-using-circle-ci-26c1b266e91f">https://betterprogramming.pub/build-a-static-website-with-jekyll-and-automatically-deploy-it-to-aws-s3-using-circle-ci-26c1b266e91f</a></li>
</ul>
<h2 id="create-github-secrets">Create GitHub Secrets</h2>
<p>Our GitHub Action will need permission to deploy to our S3 bucket and CloudFront distribution. We leverage the AWS Access Key ID and AWS Secret Access Key values to perform this. However, this is not data that we want to commit directly into our git repository or GitHub Actions file.</p>
<p>Navigate to your git repository on GitHub, then the <strong>Settings</strong> tab, and click on the <strong>New repository secret</strong> under the <strong>Secrets -&gt; Actions</strong> option.</p>
<p><img loading="lazy" src="/posts/jekyll-s3-githubactions/images/1*hF3no7OjhKfAS0MUevMqEA.webp" type="" alt=""  /></p>
<p>You will need to create three secrets.</p>
<ul>
<li>Access Key ID
<ul>
<li>Name = ‌<code>AWS_ACCESS_KEY_ID</code></li>
<li>Value = <code>Insert your value here.</code></li>
</ul>
</li>
<li>Secret Access Key
<ul>
<li>Name = ‌<code>AWS_SECRET_ACCESS_KEY</code></li>
<li>Value = <code>Insert your value here.</code></li>
</ul>
</li>
<li>CloudFront Distribution ID
<ul>
<li>Name = <code>CLOUDFRONT_DISTRIBUTION_ID</code></li>
<li>Value = <code>Insert your value here.</code></li>
</ul>
</li>
</ul>
<p>The ‌<strong>CLOUDFRONT_DISTRIBUTION_ID</strong> can be found directly under the CloudFront Distributions page in AWS Management Console.</p>
<p>We will refer to the names of both secrets in the next step.</p>
<h2 id="create-github-actions">Create GitHub Actions</h2>
<p>There are three main steps that we’ll need our GitHub Action to perform:</p>
<ol>
<li>Build the Jekyll website.</li>
<li>Sync the files to our S3 bucket.</li>
<li>Invalidate the cache of our CloudFront distribution.</li>
</ol>
<p>You likely can find a third-party action in the GitHub Action Marketplace to merge these actions, but I don’t want to introduce another dependency.</p>
<p>Navigate to your git repository on GitHub and click the <strong>New Workflow</strong> button under the <strong>Actions</strong> tab.</p>
<p><img loading="lazy" src="/posts/jekyll-s3-githubactions/images/1*N0o9ySCrUppNHYxXPvPO9Q.webp" type="" alt=""  /></p>
<p>Go ahead and click the <strong>set up a workflow yourself</strong> link and paste the following into the file:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Deploy Website</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">on</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">push</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">branches</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">main</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">jobs</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">build</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">runs-on</span><span class="p">:</span><span class="w"> </span><span class="l">ubuntu-latest</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">steps</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">actions/checkout@v2</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">ruby/setup-ruby@v1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">with</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">ruby-version</span><span class="p">:</span><span class="w"> </span><span class="s1">&#39;2.7&#39;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="l">gem install bundler</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="l">bundle install</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="l">bundle exec jekyll build</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Configure AWS Credentials</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">aws-actions/configure-aws-credentials@v1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">with</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">aws-access-key-id</span><span class="p">:</span><span class="w"> </span><span class="l">${{ secrets.AWS_ACCESS_KEY_ID }}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">aws-secret-access-key</span><span class="p">:</span><span class="w"> </span><span class="l">${{ secrets.AWS_SECRET_ACCESS_KEY }}</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">        </span><span class="nt">aws-region</span><span class="p">:</span><span class="w"> </span><span class="l">us-east-1</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Deploy to S3</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">        aws s3 sync _site s3://example.com --acl public-read --delete</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Invalidate CloudFront Cache</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">        aws cloudfront create-invalidation --distribution-id ${{ secrets.CLOUDFRONT_DISTRIBUTION_ID }} --paths &#34;/*&#34;</span><span class="w">
</span></span></span></code></pre></div><p>Under the <strong>Deploy to S3</strong> task, ensure you update <code>example.com</code> to reflect your domain. Also, update <strong>aws-region</strong> if you are running in a different region than I am.</p>
<p>Click the <strong>Start commit</strong> button.</p>
<p>If all was successful, you should see a successful build under the workflow we defined. If a step failed, you can click on the workflow run to find a build log with more details for troubleshooting.</p>
<p><img loading="lazy" src="/posts/jekyll-s3-githubactions/images/1*zZY95K9YWYbyQqQMA2_fuQ.webp" type="" alt=""  /></p>
<h1 id="conclusion">Conclusion</h1>
<p>In conclusion, setting up a static website using Jekyll, GitHub Actions, AWS S3, AWS Route 53, AWS Certificate Manager, AWS CloudFront, and AWS CloudFormation can be complex. However, by leveraging AWS CloudFormation to manage the required resources makes it much easier.</p>
<p>At this point, you should have a build pipeline that builds your Jekyll site and deploys it to AWS S3 each time you commit to the main branch. The committed changes to your website should be reflected as the build process completes successfully.</p>
<p>While I didn’t include it in this article, it’s worth noting that we could automate the creation of the IAM users, roles, and policies that are required for the GitHub Action to perform its steps. Additionally, you can modify the GitHub Action to only perform the build and deployment when a pull request is approved.</p>
<p>Lastly, don’t limit yourself to the solution I provided. There are many ways to perform the same outcome, as I’ve demonstrated here. For instance, on one of my client’s projects, I find the build process to be slower than I’d like. Rather than setting up ruby, installing bundler and installing bundles, I found that running a docker container was much quicker for me. This is because the majority of the steps defined in our original GitHub Action are cached in the <code>jekyll/builder</code> image.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">steps</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">uses</span><span class="p">:</span><span class="w"> </span><span class="l">actions/checkout@v3</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">Build the site in the jekyll/builder container</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span><span class="nt">run</span><span class="p">:</span><span class="w"> </span><span class="p">|</span><span class="sd">
</span></span></span><span class="line"><span class="cl"><span class="sd">        docker run \
</span></span></span><span class="line"><span class="cl"><span class="sd">        -v ${{ github.workspace }}:/srv/jekyll -v ${{ github.workspace }}/_site:/srv/jekyll/_site \
</span></span></span><span class="line"><span class="cl"><span class="sd">        jekyll/builder:latest /bin/bash -c &#34;chmod -R 777 /srv/jekyll &amp;&amp; jekyll build --future&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="c"># ...</span><span class="w">
</span></span></span></code></pre></div>]]></content:encoded></item><item><title>Using Docker and X11 Forwarding to run Kali Linux on macOS</title><link>https://fz42.net/posts/docker-x11-forwarding-kali/</link><pubDate>Mon, 05 Dec 2022 00:00:00 +0000</pubDate><guid>https://fz42.net/posts/docker-x11-forwarding-kali/</guid><description>Walkthrough showing how to setup Kali Linux as a Docker container and use X11 Forwarding to have a seamless workflow on macOS.</description><content:encoded><![CDATA[<p>Running Linux on macOS, but why, you say?</p>
<p><img loading="lazy" src="/posts/docker-x11-forwarding-kali/images/1*q6hwX46LXwKKkoQlmC1f7A.webp" type="" alt=""  /></p>
<h1 id="my-problem">My problem</h1>
<p>Having been an InfoSec practitioner for several years, I wouldn’t say that I’m set in my ways, but I’ve certainly developed a workflow for security assessments that I’ve grown used to. When I sat down to prepare for the Offensive Security Certified Professional (OSCP) certification, I found their prebuilt VM was greatly hindering that workflow.</p>
<p>Not only were key-bindings problematic between macOS and VMWare Fusion or VirtualBox, but I had no desire to run yet another desktop environment when macOS was perfectly capable. Furthermore, I would have to store a ~60 GB disk image on my MacBook Pro and Time Machine backups, and worry about shared drives and clipboard to make for a seamless experience. Lastly, my documentation process at the time was relying on an application that wasn’t compatible with Linux, which meant I couldn’t do everything inside the Kali VM without changing tools.</p>
<p>At first, I was going to use cloud-init, Terraform, and Ansible for this solution, but that would make more sense if I was creating a VM in my opinion. I use docker, primarily with docker-compose, in my homelab, and recalled X11 Forwarding from my old days of running Gentoo Linux as a daily driver. After some experimentation, I found this solution to be the best for my needs.</p>
<h1 id="brief-roles-of-products-in-my-solution">Brief roles of products in my solution</h1>
<h2 id="docker">Docker</h2>
<p>We will use <a href="https://www.docker.com/">Docker</a> to create an image of Kali Linux which includes any additional tools and configurations that we want.</p>
<h2 id="docker-compose">Docker-Compose</h2>
<p>We will use <a href="https://docs.docker.com/compose/">Docker Compose</a> to build our container and configure X11 Forwarding.</p>
<h2 id="xquartz">XQuartz</h2>
<p>We will use <a href="https://www.xquartz.org/">XQuartz</a> as an X Window System to display the GUIs running on Kali Linux under our macOS desktop.</p>
<h1 id="putting-it-all-together">Putting it all together</h1>
<h2 id="building-our-image">Building our image</h2>
<p>First we need to build a docker image containing Kali Linux and additional tools we want, we can create the following <code>Dockerfile</code> to handle that.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Dockerfile" data-lang="Dockerfile"><span class="line"><span class="cl"><span class="c"># We need to start from somewhere</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">FROM</span><span class="w"> </span><span class="s">kalilinux/kali-rolling:latest</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">ENV</span> DEBIAN_FRONTEND noninteractive<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="c"># Bring everything up to date</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">RUN</span> apt update<span class="p">;</span> apt -y dist-upgrade<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="c"># Install whatever we desire</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">RUN</span> apt-get -y install <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span>x11-tools <span class="se">\
</span></span></span><span class="line"><span class="cl"><span class="se"></span>zsh<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="c"># Here you can run additional commands to configure the box</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="c"># Create our new user and add it to some groups</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">RUN</span> useradd -G sudo,video fatzombi -shell /bin/zsh -m<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="c"># Create a default password so we can login</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">RUN</span> <span class="nb">echo</span> <span class="s2">&#34;fatzombi:fatzombi&#34;</span> <span class="p">|</span> chpasswd<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="c"># Clean up packages</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">RUN</span> apt-get -y autoremove<span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="c"># Jump to the shell</span><span class="err">
</span></span></span><span class="line"><span class="cl"><span class="err"></span><span class="k">ENTRYPOINT</span> zsh <span class="nv">$@</span><span class="err">
</span></span></span></code></pre></div><p>Running <code>docker build .</code> will create the image for us. If the build fails, you may need to explicitly include additional dependencies for the applications you are installing.</p>
<p><img loading="lazy" src="/posts/docker-x11-forwarding-kali/images/1*E35CBVqY6Lq163RFArqrmA.webp" type="" alt=""  /></p>
<h2 id="building-a-container-from-our-image">Building a container from our image</h2>
<p>Now we could run docker create with arguments, but in my case, I want the configuration to be stored in a <code>docker-compose.yml</code> file that I can commit to version control.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">version</span><span class="p">:</span><span class="w"> </span><span class="s2">&#34;3.7&#34;</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w"></span><span class="nt">services</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span><span class="nt">kali</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">build</span><span class="p">:</span><span class="w"> </span><span class="l">.</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">stdin_open</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w"> </span><span class="c"># docker run -i</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">tty</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">        </span><span class="c"># docker run -t</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">container_name</span><span class="p">:</span><span class="w"> </span><span class="l">kali</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">restart</span><span class="p">:</span><span class="w"> </span><span class="l">unless-stopped</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">hostname</span><span class="p">:</span><span class="w"> </span><span class="l">kali-oscp</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">environment</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">DISPLAY=host.docker.internal:0</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">TERM=xterm-256color</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">    </span><span class="nt">cap_add</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">NET_ADMIN</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">      </span>- <span class="l">SYS_ADMIN</span><span class="w">
</span></span></span></code></pre></div><p>Running <code>docker-compose up -d</code> will attempt to create the container. If successful, you should have a container called kali listed in the output of <code>docker ps</code>.</p>
<p><img loading="lazy" src="/posts/docker-x11-forwarding-kali/images/1*1CPl0MoKuymtXVtI3Ym3hg.webp" type="" alt=""  /></p>
<p>Don’t mind if your COMMAND and PORTS are different than mine, the important thing is that the container is running.</p>
<h2 id="entering-our-new-container">Entering our new container</h2>
<p>Now that our container has been started, we can enter it by running <code>docker exec --user fatzombi --workdir /home/fatzombi/ -it kali /bin/zsh</code>.
Don’t forget to use <code>passwd</code> to change the default password that was defined in the <code>Dockerfile</code>.</p>
<h2 id="displaying-guis-on-macos">Displaying GUIs on macOS</h2>
<p>All that’s left to do is start XQuartz on macOS and run the following command in the terminal to inform macOS which display should be used for X11 Forwarding.
<code>DISPLAY=:0 /opt/X11/bin/xhost +</code>
Displaying GUI applications from Kali Linux is now as simple as typing the command of the application in the terminal of the container.</p>
<p><img loading="lazy" src="/posts/docker-x11-forwarding-kali/images/1*oIaFhgjroiYXBtLm4IYimw.webp" type="" alt=""  /></p>
<h1 id="concluding-thoughts">Concluding Thoughts</h1>
<p>There are several benefits to this solution:</p>
<ul>
<li>Everything I care about can be included in source control, and I don’t need to back up a ~60 GB binary disk image.</li>
<li>We can streamline our process with Makefiles and shell scripts to increase the efficiency, see further below.</li>
<li>This solution can be applied to operating systems other than macOS. Additionally, it doesn’t require the Kali container to run locally on macOS, in case you need it to have additional resources.</li>
</ul>
<h1 id="what-else-can-we-do">What else can we do?</h1>
<h2 id="using-makefile-to-get-into-the-container-quicker">Using Makefile to get into the container quicker</h2>
<p>Add the following contents into a Makefile:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-sh" data-lang="sh"><span class="line"><span class="cl">shell run: open -g /Applications/Utilities/Xquartz.app sleep <span class="m">2</span> <span class="nv">DISPLAY</span><span class="o">=</span>:0 /opt/X11/bin/xhost + <span class="o">||</span> TRUE sleep <span class="m">2</span> docker start kali docker <span class="nb">exec</span> --user fatzombi --workdir /home/fatzombi/ -it kali /bin/zsh
</span></span></code></pre></div><p>You can now type <code>make</code> in the directory to bring yourself to the container’s shell, or you could place those commands in a shell script and create an alias to quickly enter the environment.</p>
<p>A helpful script might be to perform the image build process, handle the docker-compose and X11 Forwarding steps, and enter the container’s shell.</p>
<h2 id="mounting-volumes-from-macos-to-kali-linux">Mounting volumes from macOS to Kali Linux</h2>
<p>By adding the following lines to the <code>docker-compose.yml</code> file, it allows for you to share files between macOS and the Kali container.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">volumes</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span>- <span class="s2">&#34;/Users/fatzombi/Documents:/home/fatzombi/Documents&#34;</span><span class="w">
</span></span></span></code></pre></div><p>Using this technique, you could also share specific dotfiles between macOS and Kali Linux.</p>
<h2 id="exposing-ports">Exposing ports</h2>
<p>Depending on the assessment, I may find myself forwarding ports from one box to the other. In those situations, it’s helpful to expose ports on the container, which is just as simple as mounting volumes.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-yaml" data-lang="yaml"><span class="line"><span class="cl"><span class="nt">ports</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span>- <span class="nt">4445:4445</span><span class="p">:</span><span class="w">
</span></span></span><span class="line"><span class="cl"><span class="w">  </span>- <span class="m">16001</span><span class="p">:</span><span class="m">16001</span><span class="w">
</span></span></span></code></pre></div>]]></content:encoded></item></channel></rss>