WIP: Major content update

content/posts/2021/assisted-vectorization.md

@@ -0,0 +1,161 @@
+---
+title: "Image Vectorization 🖇🍮"
+date: 2021-12-08T19:26:46+01:00
+draft: false
+toc: true
+tags:
+  - svg
+  - python
+  - code
+  - image
+---
+
+
+Automated vectorization and upscaling is useful in many scenarios particularly
+for drawn art that can be segmented and is generally structured using strokes
+and gradients. Here I will outline a methodology that is based around
+structural analysis and direct regression methods then evaluate error metrics
+for fine-tuning by comparing the output with non-vectorized up-scalers.
+
+
+## Observations
+
+Regarding image segmentation, I suspect the most common approach is directly
+clustering the entire image using the colour and position. In
+most scenarios this feature-space will be separable and is a well understood
+problem statement. There result still poses some issues; first cluster enclosure
+is difficult to resolve (convexity is not guaranteed), second gradient
+components weaken the separability of the data. In addition we may need to
+sub-sample the image since clustering is computationally expensive given the
+mega-samples of image data.
+
+## Outline
+
+0. Coarse Analysis
+    - Cluster based on colour-space and |∇×F| |∇·F| normalized components
+    - Present image colour and segmentation complexity
+    - Partition image in delta space using histogram method
+1. Pre-processing and edge thresholding
+    - Compute the YCbCr equivalent representation
+    - Map the input colour space based on k-means / SVM for maximum cluster separation
+    - Compute edges in images and fit spline segments with grouping
+2. Fine image segmentation
+    - Use edges to initialize segments and segment regions based on colour deltas
+    - Complete segmentation by filling image
+    - Regression to fit colour for each segment
+4. Image restructuring and grouping
+    - Simplify structures by creating region hierarchy
+    - SVG 2.0 supports mesh-gradients / Bezier surface composition
+    - Detect regular patterns with auto correlation / parameter comparison
+3. Error evaluation and recalibration
+    - Use upscaled reference to evaluate error
+    - Identify segments that need to be restructured with more detail
+
+
+## Action Items
+
+ - Colour surface fitting
+    - Given a set of samples, fit an appropriate colour gradient
+ - Image normalization and pre-processing
+ - Edge composition and image segmentation
+    - Define a model for segments of the image
+    - Define a model for closed and open edges
+    - Evaluate segment coverage of the image
+ - Heuristics for image hierarchy and optimizations
+
+
+## Pre-processing Engine
+
+Currently histogram binning has proven to be very effective for generating
+initial clusters since it scales well with image size. This allows us to
+quickly gauge the complexity of an image in terms of content separability.
+These regions will be used to partition the image as edges are extracted
+before more accurate mapping is performed.
+
+The main challenge here is colour gradients that washout what should be
+obvious centroids. Analysing the samples in batches can prevent this to some
+extent but a more robust approach is needed. We could use a localized grouping
+technique but accuracy may need be that important for this pre-processing step.
+Another technique is that the image can first be clustered using the histogram
+of derivative components followed by sub-classing a histogram for each gradient
+cluster.
+
+This idea for histogram-binning is surprisingly efficiently for artificial
+images where the colour pallet is rich in features. A binary search for
+parameterizing the local maxima detection will very quickly segment a wide
+variety of images into 10 - 30 classifications.
+
+## Edge extraction
+
+At some point we will want to construct a model representing regions and shapes.
+The principle component here is identifying edges segmenting the image. Edge
+detection is relatively strait-forward as we only need look for extrema in the
+derivative components. In most scenarios this is actually quite noisy and
+it is not obvious how we should threshold for what is and what is not an edge.
+
+Here we use the histogram-based clustering result for edge detection region to
+region transitions are discretized and no adaptive threshold is required.
+There will unavoidably be noisy regions where we see this boundary being spread
+out or possibly just a select few pixels appearing in some sub-section due to
+the clustering process. This can mostly be removed with a median filter if
+necessary.
+
+If the initial segmentation are generated based on k-means in the colour space,
+two families of edges will be detected along segments: hard and soft edges.
+Hard edges will correspond to the intended edges seen in the image where as
+soft edges will arise due to the clustering technique. We can classify these
+two families by looking at the norm of the derivative component along such an
+edge. There will be more than one way to asses the correctness here but the
+significance here is that soft edges present boundary conditions during colour
+mapping while hard edges do not. Otherwise visual artefacts will arise are the
+interface of two segments that originally was a smooth transition.
+
+
+## Structural Variability
+
+While we are targeting a-typical images for vectorization it is obvious that
+'sharpness' in the final result depends on a subjective style that is difficult
+to assess in terms of simple regress or interpolation. This problem however is
+central to upscaling algorithms so the methodology here will be that a external
+upscaling tool will guide the vectorization process. For example vectorizing
+a pixel-art image yields better results 'nearest-neighbour' methods opposed to
+Lanczos resampling.
+
+
+## Regression over SVG colour surfaces
+
+The SVG standard supports three methods for specifying colour profiles or
+gradients: Flat, Linear, Radial. There are more advanced mechanisms through
+embedding or meshing multiple components the aforementioned three readily
+allow us to fit a first order colour contour through linear
+regression. This will be our first objective for parameterizing
+the colour for segments in our image. Another thing to note is that the gradient
+can be transformed after being parameterized. This means that the a circular
+gradient can be transformed to realize a family elliptical gradients.
+
+Obviously this will not accommodate all colour contours that we find in images
+but in such scenarios we may adopt piece-wise approximations or more accurate
+masking of each component using the alpha channel. At some point we should
+also be able to resolve mixtures and decompose the contour from a non-linear
+or higher-order surface into multiple simpler contours. Again note that support
+for advanced colour profiles is not well supported so composition through
+these basic elements will yield the best support.
+
+Using linear regression here with a second order polynomial kernel is a very
+efficient method for directly quantifying the focal point of the colour
+gradient if there is one.
+
+
+## Contour estimation
+
+My initial attempt to estimate the contour given a set of points was based on
+using the convex-hull and recursively enclosing the outline in more detail
+by interpolating in between the current outline and finding the closest point
+orthogonal to the outline. This result yields a fast approximation of the
+enclosing outline without many requirements on the set of points other than
+having a dense outline. The drawback was that if it difficult to detect
+incorrect interpolation and only resolves the outline with pixel-level
+precision. If we pre-process the collection of points such that they
+represent detected edges at sub-pixel resolution the later draw-back can be
+addressed. Correctness or hypothesis testing could yield a more robust result
+at the cost of increased complexity.

content/posts/2021/building-svg.md

@@ -87,7 +87,7 @@ added directly to KGT as a feature in future releases.
 
 The final result is shown below.
 
-![example_kgt.svg](/images/example_kgt.svg)
+{{< figure src="/images/posts/example_kgt.svg" title="example_kgt.svg" >}}
 
 ## Tabatkins Railroad Diagrams
 
@@ -118,15 +118,29 @@ would look like this:
 
 ``` python
 import railroad
-with open("./test.svg","w+") as file:
+with open("./posts/test.svg","w+") as file:
     obj = railroad.Diagram("foo", railroad.Choice(0, "bar", "baz"), css=style)
     obj.writeSvg(file.write)
 ```
 
 The final result is shown below.
 
-![example_kgt.svg](/images/example_trd.svg)
+{{< figure src="/images/posts/example_trd.svg" title="example_trd.svg" >}}
 
 Note that this figure is quite a bit more compact but adding additional labels
 or customizations outside the scope of the library will probably require
 quite a bit of manual work. This could be a fun side project though.
+
+# Using Hugo Short Codes
+
+
+
+``` go
+{< python-svg dest="/images/posts/test.svg" title="This is a pyuthon-svg exmaple." >}
+railroad.Diagram("foo", railroad.Choice(0, "bar", "baz"), css=style)
+{< /python-svg >}
+```
+
+{{< python-svg dest="/images/posts/test.svg" title="This is a python-svg exmaple." >}}
+railroad.Diagram("foo", railroad.Choice(0, "bar", "baz"), css=style)
+{{< /python-svg >}}

content/posts/2021/configure-nginx.md

@@ -1,9 +1,8 @@
 ---
 title: "Setting up a NGINX configuration 🧩"
 date: 2021-10-31T15:08:33+01:00
-draft: false
+draft: true
 toc: false
-images:
 tags:
   - website
   - config

content/posts/2021/mermaid-uml.md

@@ -42,7 +42,7 @@ graph LR
 
 This example generates the diagram show below.
 
-![example_mermaid.svg](/images/example_mermaid.svg)
+{{< figure src="/images/posts/example_mermaid.svg" title="example_mermaid.svg" >}}
 
 There are four base themes: dark, default, forest, neutral. Additional
 [customization](https://mermaid-js.github.io/mermaid/#/theming) is possible.
@@ -73,7 +73,7 @@ diagrams of classes and inter-related structures. For example the UML diagram be
 [pyviewer]({{< relref "pyside.md" >}} "pyside") which is image simple
 browsing utility for compressed archives.
 
-![example_pyviewer.svg](/images/example_pyviewer.svg)
+{{< figure src="/images/posts/example_pyviewer.svg" title="example_pyviewer.svg" >}}
 
 This does quite well at illustrating how classes are composed and which methods
 are available at various scopes. It also helps organizing and structuring a
@@ -153,7 +153,7 @@ function main() {
   esac
   # echo "IN:${ARGS[1]}  OUT:${ARGS[3]}"
   mmdc ${ARGS[@]} &> /dev/null
-  mogrify -trim "${ARGS[3]}" 
+  mogrify -trim "${ARGS[3]}"
   feh --reload 2 "${ARGS[3]}" &
   sleep 0.1
   inotifywait -qm --event modify --format '%w' "${ARGS[1]}" | \

content/posts/2021/my-2018-setup.md

@@ -0,0 +1,38 @@
+---
+title: "My 2018 Setup"
+date: 2021-08-12T10:24:27+02:00
+draft: false
+toc: true
+tags:
+  - website
+  - about
+---
+
+I mainly use RHEL flavours of linux having both CentOS and Fedora machines. Most
+hosted services run on CentOS 8 at the moment albeit they are approaching
+end-of-life. Overall the package repository for CentOS 7/8 is just right. I
+rarely need to compile anything from source and packages are very stable.
+I will eventually migrate to Fedora completely which is where I operate my
+development environment.
+
+This is a list of my most used self-hosted services:
+ - Gitea: Git server with web interface for repository mirrors and personal repos
+ - Plex: multi-media hosting service for streaming movies and tv-shows
+ - NextCloud: Cloud storage for synchronizing and sharing files
+ - Cockpit: Web base administration portal managing linux boxes
+ - RoundCube: Web based email client
+ - Postfix/Dovcot: Email stack providing SMTP for my domain
+ - NGINX: HTTP server serving as proxy for internal web services
+ - Danbooru: Ruby-on-rails based image hosting and tagging service
+
+There are several others that I have tried but these really have been the things
+I relied on the most in the past 5 years or so. I think the only thing that is
+possibly missing from this list is possibly the equivalent of a centralized LDAP
+service but I simply haven't had to manage more than handful of users.
+
+Currently I develop quite a bit of python utilities for scraping, labelling, and
+managing media in an automated fashion. In part I am preparing data for one of
+my long term projects which is related to image classification based on
+structural decomposition rather than textural features. The main idea here is
+to analyse and extract structure in an image before performing in-depth analysis
+such that said analysis is most specific to its context.

content/posts/2021/pyside.md

@@ -64,18 +64,62 @@ to a qml function call "swipe.update_paths" for example.
 viewer.path_changed.connect(swipe.update_paths)
 ```
 
+## Example: passing images as bindary data
+
+For reference the code below outlines a simple example that loads an image from
+a zip archive and makes the binary data available for QML to source. This
+avoids the need for explicit file handles when generating or deflating images
+that are needed for the QML front-end.
+
+```python
+class Archive(ZipFile):
+    """Simple archive handler for loading data."""
+    @property
+    def binarydata(self) -> bytes:
+        """Load file from archive by name."""
+        with self.open(self.source_file, "r") as file:
+            return file.read()
+```
+
+The example class above simply inherits from the zipfile standard library where
+we read a image and store it as part of the `PyViewer` class shown below. This
+class inherits from `QObject` such that the property is exposed to the qml
+interface. In this case the `imageloader` is an `Archive` handler that is
+shown above.
+
+```python
+class PyViewer(QObject):
+    """QObject for binging user interface to python backend."""
+    @Property(QByteArray)
+    def image(self) -> QByteArray:
+        """Return an image at index."""
+        return QByteArray(self.imageloader.binarydata).toBase64()
+```
+
+This setup allows a relatively clean call to the `viewer.image` property within
+the QML context as shown below. Other data types such as `int`, `string`,
+`float`, and booleans can be passed as expected without requiring the
+QByteArray container.
+
+```qml
+Image {
+  anchors.fill: parent
+  fillMode: Image.PreserveAspectFit
+  mipmap: true
+  source = "data:image;base64," + viewer.image
+}
+```
+
 ## Downside
 
 Debugging and designing QML in this environment is limited since the pyside
 python library does not support all available QML/QT6 functionality. In most
 cases you are looking at C++ Qt documentation for how the pyside data-types
-and methods are supposed to behave without good hinting.
+and methods are supposed to behave without good hinting. Having developed
+native C++/QML projects previously helps a lot. The main advantage here is t
+hat QML source code / frame-works can be reused.
 
-Also the variety in data types that can be passed from one context to the other
-is constrained although in this case I was able to manage with strings and byte
-objects.
-
-## Other Notes: TODO
+## Other Notes:
 
 ```python
 ImageCms.profileToProfile(img, 'USWebCoatedSWOP.icc',

content/posts/2021/super_resolution.md

@@ -0,0 +1,21 @@
+---
+title: "Super Resolution 🧙‍♂️"
+date: 2021-09-19T13:30:00+02:00
+draft: true
+toc: true
+math: true
+tags:
+  - upscaling
+  - image-processing
+  - anime
+  - python
+---
+
+WIP: this is an on going effort for super-resolving images given learned context
+and Super-Resolution Using a Generative Adversarial Network (SRGAN). [^1]
+
+$$ y_t = \beta_0 + \beta_1 x_t + \epsilon_t $$
+
+now inline math \\( x + y \\) =]
+
+[^1]: And that's the footnote.