Generative AI Model Bending Model Crafting Diffusion ComfyUI

This project implements model bending techniques for diffusion models in ComfyUI. Model bending allows you to apply transformations to the inner workings of your model in order to push it towards new and diverse aesthetics. You can see it as providing granular controls for introducing diversity and randomization beyond your simple randomization seeds. Transformations can include addition, multiplication, noise, rotation, erosion, dilation, to name a few, ... or even your own custom ones.

Generative AI Design Fiction Future

A fictional website with promotional material for an AI augmentation product. The was created to imagine a future where AI models are treated as commodities. The podcast, created with Google's Notebook, presents two speakers discussing the implications of the technology.

Data Visualization Surrogate Modelling Building Performance Experts Review Web Team Project

The tool leverages a surrogate Ml-based model for daylighting prediction that reduces heavy simulations. The goal was to create a tool that makes advanced performance calculations accessible to designers in the early design stages with not much building science experience.

Role: included everything from proposal writing, literature review, problem identification and gathering requirements, project management (Gantt), technical feasibility testing, mockup design (Figma), prototyping and full-stack web development (Vue.js, CefSharp, D3.js, Grasshopper, Rhino Compute), designing and conducting studies to evaluate the tool, and analyzing and presenting study's data, writing papers and presenting to the team and at international conferences.

Data Visualization Design Study Web Team Project

We conducted a design study to explore the applicability of data visualizations for supporting Dungeon Masters in Dungeons and Dragons (D&D) games. We interviewed Dungeon Masters and arrived at multiple visualization modules tackling different needs. One of which is a tool for monster shopping.

An essential part of a D&D session is fighting encounters with monsters. Dungeon Masters (DMS) have to choose monsters that are sufficiently challenging to the players and which are consistent with the setting and the story. Some DMs use physical manuals to look for monsters. While it can be engaging and inspiring to leaf through pages full of pictures, physical manuals lack the filtering mechanisms afforded by digital tools. On the other hand, some DMs use online interfaces (e.g., DnDBeyond) to quickly filter monsters by different attributes like their challenge rating or type but those interfaces do not foster serendipitous encounters or encourage visual exploration.

The Monster Shopper is an attempt for finding a balance between visual and data-driven exploration of options for novice users. DesignSense (below) is aimed at professional designers who have higher analytical needs such as identifying statistical patterns. The Monster Shopper is built as an adaptation and improvement upon a system by the Science Visualization Group at AMNH.

Role: Research, design, and development.

Data Visualization Generative Design Dissertation Vue.js D3.js

Rhino3D Grasshopper Performance-Based Design Team Project

FlowUI is a design-aid tool that provides immediate feedback to architectural designers about the performance of their directly-modeled designs through interactive visualizations and analysis.

PCG Answer Set Programming Graph Partitioning

Taksim is a procedural generation system for generating game spaces that adheres to a set of high and low level design constraints. Mainly, being able to dictate which regions in that space should be adjacent or inadjacent. This has many applications like generating a dungeon with a predetermined sequence of in-game missions or generating political maps (e.g., Risk).

Taksim uses a novel approach by dressing up the problem as a constrained graph partitioning and solving it using Answer Set Programming. The result is a highly flexible system.

Publications on Taksim can be found in the publications page and this is the link to my confernece presentation on Taksim.

Processing Generative Art

Inspired by a gif of an aging woman, I wanted to find a way to juxtapose the images from different ages together. In JuxtaPause, I stack the gif's frames as a 3D matrix (movie?) then sample across the matrix using a greyscale 2D Perlin Noise image. This was favoured over using an RNG because Perlin noise created contiguous chunks of light/dark regions which made it possible to have just enough of each age. Moving the mouse around directly manipulates two parameters of the Perlin noise. Updates: you can now connect a midi controller to control 4 parameters, and the piece uses p5.js instead of Processing/Java. I have also included 2 more experiments!

Java OpenCV

When looking at the Noisy GIF project above, you can explore its generative space by moving the mouse around looking for something 'interesting'. I was interested in automating/semi-automating this process. An optimization algorithm fails in this case because it is hard to quantify interestingness. Djinn aims at supporting the process of exploring the space of a visual art generator by sampling it through Novelty Search. The purpose is to find as many distinct/novel points in that space as possible. This is done by comparing any newly generated piece against others based on their visual features (I am using Histogram of Oriented Gradients (HOG)).