Welcome to the world of Art merged with Technology ✨

Ari Melenciano
9 min readApr 30, 2019

--

There’s an amazing world that lies at the intersections of art and technology. This interdisciplinary field merges the arts, design, technology, engineering and humanities to introduce a great variety of possibilities that further extend everyday life. This guide is meant to be a quick introduction to different creative technology tools and resources that allow and teach innovation across a multitude of emerging mediums.This guide has been developed by Ari Melenciano, an NYU Interactive Telecommunications Graduate Program (ITP) Research Fellow. This guide was created with the support of University of Denver’s Clinic for Open Source Arts as Ari was the first Contributor-In-Residence.

Her work through this residency focused on making open source and creative technologies more accessible to people of different racial, cultural, gender, and socio-economical backgrounds. Ari is especially passionate about empowering future Black, Latino/a and women technologists, all of which are under-represented groups in the technology field. She is also the founder of Afrotectopia, a new media arts, culture and technology festival, and developed a youtube channel to share the possibilities of art and technology.

Before we explore all of the exciting ways technology and art can merge to realize new realities, it’s important that there is an understanding of how technology and the humanities always operate, hand-in-hand. Technology provides exceptional convenience, and can augment every day life to great moments of awe. But, the convenience technology provides is through an exchange of personal data; the role technology plays in every day life has also often exhibited incredibly harmful effects to marginalized communities, and an awareness of how technology can continue cycles of oppression is vital to developing sustainable and ethical innovations. Below are a few resources to learn more, though there are many many more.

Books:

Weapons of Math Destruction by Cathy O’Neil

Automating Inequality by Virginia Eubanks

Carceral Capitalism by Jackie Wang

Surveillance Valley, The Secret Military History of the Internet by Yasha Levine

Institutions:

Data and Society

AI Now

Institute for Public Knowledge

Let us begin!

1. What is Creative Technology?

Creative technology is a product of the combined super powers of art and technology. Artistic expression has reached new potentials as technology grows in capability. The art advances the experiences of technology, while the technology advances the possibilities of the art. Creative technology can take many different forms, from human-computer interactivity, installations, products, to websites and beyond.

Some common places you might have experienced creative technology is SnapChat filters that have detected the different parts of your face and overlay a perfectly mapped augmented addition (i.e. makeup, flower crowns, etc.). The incredible set designs of Beyoncé concerts? Creative technology. Pretty much any app on your phone, but especially those that allow you to express artistically: Instagram, Tumblr, VSCO.

2. Different creative technology mediums:

Creative Computation/Coding/Programming:

Using code for human-computer interactive purposes, digital media, music, data visualization, animation, and a wide variety of other artistic practices.

Languages:

JavaScript: the language of web browsers, including HTML5. JavaScript is a prototype-based, multi-paradigm, dynamic language, supporting object-oriented, imperative, and declarative (e.g. functional programming) styles.

C++: a general-purpose programming language that has imperative, object-oriented and generic programming features, while also providing facilities for low-level memory manipulation.

Python: an interpreted, high-level, general-purpose programming language.

Swift: a general-purpose, multi-paradigm, compiled programming language developed by Apple Inc. for iOS, macOS, watchOS, tvOS, Linux, and z/OS.

Platforms & Libraries:

Processing: a flexible software sketchbook and a language for learning how to code within the context of the visual arts. Processing has promoted software literacy, particularly within the visual arts, and visual literacy within technology. Initially created to serve as a software sketchbook and to teach programming fundamentals within a visual context, Processing has also evolved into a development tool for professionals. The Processing software is free and open source, and runs on the Mac, Windows, and GNU/Linux platforms.

Processing (via https://www.eventbrite.ca/e/drawing-games-an-intro-to-programming-with-processingorg-tickets-58455140900)

| Tutorials by Dan Shiffman.

p5.js: a Javascript library that starts with the original goal of Processing, to make coding more accessible for artists, designers, educators, and beginners, and reinterprets this for today’s web.

p5js (via https://medium.com/processing-foundation/hello-p5-js-web-editor-b90b902b74cf)

| Tutorials by Dan Shiffman and Ari Melenciano

Scratch: With Scratch, you can program your own interactive stories, games, and animations — and share your creations with others in the online community. Scratch helps young people learn to think creatively, reason systematically, and work collaboratively — essential skills for life in the 21st century. Scratch is a project of the Lifelong Kindergarten Group at the MIT Media Lab. It is provided free of charge.

Scratch by MIT

Glitch: From useful tools that solve problems at work, to cutting-edge VR experiences, smart bots, and apps that help advance important causes, Glitch is a unique community where people have built over a million projects for you to discover, with new ones are popping up every day.

Glitch (via https://blog.bitsrc.io/introduction-to-glitch-for-node-js-apps-in-the-cloud-cd263de5683f)

OpenFrameworks: an open source C++ toolkit designed to assist the creative process by providing a simple and intuitive framework for experimentation.

OpenFrameworks (via https://openframeworks.cc/about/)

Cinder: a free and open source library for professional-quality creative coding in C++.

Cinder (via https://www.youtube.com/watch?v=Fx3hTgrOyFg)

Physical computation:

Bringing a variety of different sensors together, through electrical engineering techniques, to create something new.

Arduino: an open-source electronics platform based on easy-to-use hardware and software. It’s intended for anyone making interactive projects.

Arduino (via https://www.jsumo.com/arduino-mega-advanced-kit-original-mega)

Raspberry Pi: a small and affordable computer that you can use to learn programming.

Raspberry Pi (via https://www.makeuseof.com/tag/5-wearable-projects-can-build-raspberry-pi/)

Little Bits: easy and versatile 21st Century tool for invention-based learning. The ever-growing library of modular snap-together Bits are gender-neutral, age agnostic, and independent of experience or ability.

littleBits ( via https://www.littlebits.cc/)

Bare Conductive: a suite of printed electronics products that enable individuals and companies to integrate electronics directly into the environment. Their conductive paint and easy-to-use development kits let anyone prototype their electronic visions of the future.

Bare Conductive (via https://www.kickstarter.com/projects/863853574/electric-paint-lamp-kit-paint-plug-and-play)

Leap Motion: Reach into the future of virtual and augmented reality with the most advanced hand tracking on Earth, used by over 300000 developers worldwide.

Leap Motion

Virtual and Augmented Reality + Game Design:

Unity: a cross-platform real-time engine and can be used to create both three-dimensional and two-dimensional games as well as simulations for its many platforms.

Unity (via https://www.youtube.com/watch?v=PC2mjd8su3I)

Vuforia: an augmented reality software development kit (SDK) for mobile devices that enables the creation of augmented reality applications. It uses computer vision technology to recognize and track planar images (Image Targets) and simple 3D objects, such as boxes, in real time.

Vuforia (via https://www.itgenerator.com/augmented-reality-app-development/)

Unreal: a game engine with its code written in C++.

Unreal (via https://studybreaks.com/tvfilm/unreal-engine-epic-games/)

Visual Programming + Sound/Video Synthesis:

MaxMSP: Max connects objects with virtual patch cords to create interactive sounds, graphics, and custom effects.

MAXMSP from Cycling74 projects page

Pure Data: a real-time graphical programming environment for audio, video, and graphical processing.

Pure Data (http://idadon.com/projects/pure-data/)

3D Rendering:

Blender: a free and open-source 3D computer graphics software toolset used for creating animated films, visual effects, art, 3D printed models, interactive 3D applications and video games.

Blender (https://www.blender.org/download/demo-files/)

Maya: is a 3D computer graphics application that runs on Windows, macOS and Linux, (currently owned and developed by Autodesk, Inc.)

Maya (via https://www.youtube.com/watch?v=KMpME1yhLK8)

TinkerCAD: (aka Autodesk 123D) is a suite of hobbyist CAD and 3D modeling tools created by Autodesk. It is similar in scope to Trimble SketchUp and is based on Autodesk Inventor. As well as the more basic drawing and modeling capabilities it also has assembly and constraint support and STL export.

Tinkercad (via https://all3dp.com/1/best-free-3d-printing-software-3d-printer-program/)

Rhino3D: is a commercial 3D computer graphics and computer-aided design application.

Rhino3D (via https://www.youtube.com/watch?v=yXJA3UK9Wgw)

Animation:

Adobe Fuse: The new 3D modeling app lets you quickly create unique human characters for your Adobe Photoshop CC images, designs, prototypes and more.

Adobe Fuse (Ari Melenciano)

Mixamo: Animated 3D characters. No 3D knowledge required.
Rapidly create, rig and animate unique characters for design projects.

Mixamo (https://steemhunt.com/tag/animation/@sharmaprashant/mixamo-design-rig-and-animate-3d-characters)

Machine Learning:

Wekinator: a free, open source software that allows anyone to use machine learning to build new musical instruments, gestural game controllers, computer vision or computer listening systems, and more.

Wekinator (https://www.instructables.com/id/Using-Machine-Learning-and-YOUR-FACE-to-Control-Sc/)

ML5: Friendly machine learning for the web. ml5.js aims to make machine learning approachable for a broad audience of artists, creative coders, and students. The library provides access to machine learning algorithms and models in the browser, building on top of TensorFlow.js with no other external dependencies.

ML5 (https://ml5js.org/)

Runway: Artificial Intelligence for Augmented Creativity. Runway is a toolkit that allows creators of all kinds to use artificial intelligence in an intuitive way.

Runway (https://blog.pocketcluster.io/author/stkim1/page/6/)

These are very few out of many more programs, softwares, languages, platforms, etc that merge art and technology. If you are interested in diving deeper, or generally being exposed to technologies not listed above, take a look at the links below:

Resources:

Please be mindful these are very short lists of many possible options

Examples of projects:

Experiments with Google

CodePen.io

Creative Applications Network

Institutions doing Creative Tech:

Schools:

NYU ITP (Interactive Telecommunications)

University of Denver’s Clinic for Open Source Arts

M.I.T. Media Lab

Parson’s Design and Technology

School of Poetic Computation

Residencies/Incubators:

NewINC

Eyebeam

New Frontier Lab

rLab

LACMA

Pioneer Works

NewLab

ThoughtWorks

Autodesk

CultureHub

Mana Contemporary

A list of creative technology lists:

New York University’s Interactive Telecommunications Program Course List

“Awesome-Creative-Coding” repository on Github by TerkelG

“Awesome” repository on Github by Sindre Sorhus

Where to buy creative tech equipment:

Adafruit

Tinkersphere

Have fun, get ready to be challenged, and know that with persistence it will be so rewarding. In my first semester at ITP, I struggled grasping many of the concepts of creative coding and physical computation. But as I built projects that merged my different passions (from music and developing sound interactive technologies, to photography and building my own camera), I learned so much, so quickly. And so will you!

Wishing you all the best,

Ari

Thank you many times, again, University of Denver’s Clinic for Open Source Arts for supporting my work in making technology a more inclusive space.

--

--