Building the systems behind Muse.
Muse requires translating visual information about clothing into structured information software can work with — at the intersection of application development, image processing, computer vision, AI and visual product recognition.
Application
In developmentThe software layer through which users interact with Muse.
Images
In developmentImages are the fundamental input, because clothing enters the system through photography.
Computer vision
ExploredTechnologies for understanding clothing imagery.
AI
ExploredHigher-level interpretation beyond simple image processing.
Product recognition
ExploredIdentifying garments from images and connecting them to product information.
Search
PlannedRetrieval becomes increasingly important as the wardrobe grows.
Data
Open questionStructured information about garments and the relationships between them.
Not every technology investigated during development necessarily becomes part of the final architecture. Technical decisions are therefore documented separately from experiments.
Conceptual architecture — evolving.
This is a conceptual model of how information could flow through Muse. It is not a claim that this architecture has been implemented.
Teaching Muse to understand clothing.
A photograph contains much more than a product identity. Muse ultimately needs to reason about the visual characteristics that make a garment useful inside a wardrobe.
Questions being explored
- How accurately can clothing be recognized from photographs?
- Which garment attributes can be extracted automatically?
- How should visually similar garments be represented?
- When is generic computer vision sufficient, and when is a specialized model necessary?
- How can AI-generated information be validated?
- How should users correct incorrect recognition?
Google Cloud exploration Explored
Google Cloud's computer-vision capabilities were investigated in connection with Muse, including Cloud Vision and Product Search concepts for clothing imagery. A Google Cloud environment was set up during this exploration.
The purpose was to understand whether existing visual-recognition infrastructure could provide useful building blocks for clothing and product recognition.
Vision API versus specialized systems
An important distinction that emerged is that general-purpose image understanding and product-specific visual search are different problems. General vision capabilities help interpret an image; product-search systems are oriented toward identifying or retrieving products by visual similarity.
This matters because Muse is not simply asking "is there a shirt in this image?" — it is potentially asking "what is this garment, what does it look like, and how does it relate to the rest of this person's wardrobe?"
Turning photographs into useful inputs.
The image-processing layer sits between the user's photograph and higher-level understanding. The same garment can look substantially different depending on lighting, background, camera angle, distance, wrinkles, and whether it is worn or photographed separately.
A robust Muse experience therefore depends not only on the model but on the quality and consistency of its visual inputs.
clothing isolation
image quality
background handling
garment visibility
visual features
image representations
similarity
recognition
Specific image-processing techniques will be documented here as they are implemented or experimentally tested. Muse does not yet have a finalized computer-vision pipeline, and this page will not claim one until it does.
Connecting visual information with searchable product information.
One direction explored for Muse is visual product recognition: comparing a photograph of clothing against a structured collection of products or visual representations.
Google Cloud Product Search was investigated as a possible technology for clothing imagery, and a product-set approach for clothing images was considered.
Open technical questions
- How well does generic product recognition work for fashion?
- How should clothing products be represented?
- Is exact identification necessary, or is visual similarity more useful?
- How should unknown garments be handled?
- What happens when the user's garment is not commercially available?
- What information should be stored locally versus retrieved externally?
These should become experiments rather than assumptions — see the Lab.
Recorded separately from experiments.
Technical decisions are recorded here once made, using the same format as product decisions: context, options, reasoning, trade-offs, status. Problems and their solutions are logged chronologically in the project log.