If You Can’t Beat Them, Join Them: How Text-to-Image Tools Can Be Leveraged in the 3D Modelling Process

Samuel Otto Mathiesen; Alessandro Canossa

doi:10.1007/978-3-031-48057-7_11

If You Can’t Beat Them, Join Them: How Text-to-Image Tools Can Be Leveraged in the 3D Modelling Process

Samuel Otto Mathiesen, Alessandro Canossa

Institut for Visuelt Design

Publikation: Bidrag til bog/antologi/rapport › Bidrag til bog/antologi › Forskning › peer review

Abstract

The rapidly growing field of text-to-image generation leverages machine learning and natural language processing to allow for the creation of very diverse and high-quality images. Whilst these images can be used in various forms and for many purposes, this paper addresses a specific question: The possibility of utilizing text-to-image generators to create concept art for the 3D-modeling process of a character.
There is a potential opportunity to explore how these tools can be incorporated in traditional content creators’ pipelines. This opportunity presents itself now because of two factors: The increased capability of these text-to-image tools compared to previous generative models, as well as the lack of specific focus on creators in current research.
We propose a possible pipeline to incorporate these new tools in the traditional 3D modeling process, dubbed the ‘vortex process’.

Originalsprog	Engelsk
Titel	HCI International 2023 – Late Breaking Papers : 25th International Conference on Human-Computer Interaction, HCII 2023, Copenhagen, Denmark, July 23–28, 2023, Proceedings, Part VI
Redaktører	Helmut Degen, Stavroula Ntoa, Abbas Moallem
Antal sider	19
Forlag	Springer
Publikationsdato	23 jul. 2023
Sider	162-181
DOI	https://doi.org/10.1007/978-3-031-48057-7_11
Status	Udgivet - 23 jul. 2023
Begivenhed	HCI International 2023: 25th International Conference on Human-Computer Interaction - AC Bella Sky Hotel and Bella Center, Copenhagen, Danmark Varighed: 23 jul. 2023 → 28 jul. 2023 Konferencens nummer: 25 https://2023.hci.international/

Konference

Konference	HCI International 2023
Nummer	25
Lokation	AC Bella Sky Hotel and Bella Center
Land/Område	Danmark
By	Copenhagen
Periode	23/07/2023 → 28/07/2023
Internetadresse	https://2023.hci.international/

Navn	Lecture Notes in Computer Science
ISSN	0302-9743

Kunstnerisk udviklingsvirksomhed (KUV)

Ja

Adgang til dokumentet

10.1007/978-3-031-48057-7_11

Citationsformater

Mathiesen, S. O., & Canossa, A. (2023). If You Can’t Beat Them, Join Them: How Text-to-Image Tools Can Be Leveraged in the 3D Modelling Process. I H. Degen, S. Ntoa, & A. Moallem (red.), HCI International 2023 – Late Breaking Papers: 25th International Conference on Human-Computer Interaction, HCII 2023, Copenhagen, Denmark, July 23–28, 2023, Proceedings, Part VI (s. 162-181). Springer. https://doi.org/10.1007/978-3-031-48057-7_11

Mathiesen, Samuel Otto ; Canossa, Alessandro. / If You Can’t Beat Them, Join Them : How Text-to-Image Tools Can Be Leveraged in the 3D Modelling Process. HCI International 2023 – Late Breaking Papers: 25th International Conference on Human-Computer Interaction, HCII 2023, Copenhagen, Denmark, July 23–28, 2023, Proceedings, Part VI. red. / Helmut Degen ; Stavroula Ntoa ; Abbas Moallem. Springer, 2023. s. 162-181 (Lecture Notes in Computer Science).

@inbook{d65add103ff94c859e0cd17d44eb600e,

title = "If You Can{\textquoteright}t Beat Them, Join Them: How Text-to-Image Tools Can Be Leveraged in the 3D Modelling Process",

abstract = "The rapidly growing field of text-to-image generation leverages machine learning and natural language processing to allow for the creation of very diverse and high-quality images. Whilst these images can be used in various forms and for many purposes, this paper addresses a specific question: The possibility of utilizing text-to-image generators to create concept art for the 3D-modeling process of a character.There is a potential opportunity to explore how these tools can be incorporated in traditional content creators{\textquoteright} pipelines. This opportunity presents itself now because of two factors: The increased capability of these text-to-image tools compared to previous generative models, as well as the lack of specific focus on creators in current research.We propose a possible pipeline to incorporate these new tools in the traditional 3D modeling process, dubbed the {\textquoteleft}vortex process{\textquoteright}. ",

author = "Mathiesen, {Samuel Otto} and Alessandro Canossa",

year = "2023",

month = jul,

day = "23",

doi = "10.1007/978-3-031-48057-7_11",

language = "English",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "162--181",

editor = "Helmut Degen and Stavroula Ntoa and Abbas Moallem",

booktitle = "HCI International 2023 – Late Breaking Papers",

note = "HCI International 2023 : 25th International Conference on Human-Computer Interaction, HCII2023 ; Conference date: 23-07-2023 Through 28-07-2023",

url = "https://2023.hci.international/",

}

Mathiesen, SO & Canossa, A 2023, If You Can’t Beat Them, Join Them: How Text-to-Image Tools Can Be Leveraged in the 3D Modelling Process. i H Degen, S Ntoa & A Moallem (red), HCI International 2023 – Late Breaking Papers: 25th International Conference on Human-Computer Interaction, HCII 2023, Copenhagen, Denmark, July 23–28, 2023, Proceedings, Part VI. Springer, Lecture Notes in Computer Science, s. 162-181, HCI International 2023, Copenhagen, Danmark, 23/07/2023. https://doi.org/10.1007/978-3-031-48057-7_11

If You Can’t Beat Them, Join Them: How Text-to-Image Tools Can Be Leveraged in the 3D Modelling Process. / Mathiesen, Samuel Otto; Canossa, Alessandro.
HCI International 2023 – Late Breaking Papers: 25th International Conference on Human-Computer Interaction, HCII 2023, Copenhagen, Denmark, July 23–28, 2023, Proceedings, Part VI. red. / Helmut Degen; Stavroula Ntoa; Abbas Moallem. Springer, 2023. s. 162-181 (Lecture Notes in Computer Science).

Publikation: Bidrag til bog/antologi/rapport › Bidrag til bog/antologi › Forskning › peer review

TY - CHAP

T1 - If You Can’t Beat Them, Join Them

T2 - HCI International 2023

AU - Mathiesen, Samuel Otto

AU - Canossa, Alessandro

N1 - Conference code: 25

PY - 2023/7/23

Y1 - 2023/7/23

N2 - The rapidly growing field of text-to-image generation leverages machine learning and natural language processing to allow for the creation of very diverse and high-quality images. Whilst these images can be used in various forms and for many purposes, this paper addresses a specific question: The possibility of utilizing text-to-image generators to create concept art for the 3D-modeling process of a character.There is a potential opportunity to explore how these tools can be incorporated in traditional content creators’ pipelines. This opportunity presents itself now because of two factors: The increased capability of these text-to-image tools compared to previous generative models, as well as the lack of specific focus on creators in current research.We propose a possible pipeline to incorporate these new tools in the traditional 3D modeling process, dubbed the ‘vortex process’.

AB - The rapidly growing field of text-to-image generation leverages machine learning and natural language processing to allow for the creation of very diverse and high-quality images. Whilst these images can be used in various forms and for many purposes, this paper addresses a specific question: The possibility of utilizing text-to-image generators to create concept art for the 3D-modeling process of a character.There is a potential opportunity to explore how these tools can be incorporated in traditional content creators’ pipelines. This opportunity presents itself now because of two factors: The increased capability of these text-to-image tools compared to previous generative models, as well as the lack of specific focus on creators in current research.We propose a possible pipeline to incorporate these new tools in the traditional 3D modeling process, dubbed the ‘vortex process’.

U2 - 10.1007/978-3-031-48057-7_11

DO - 10.1007/978-3-031-48057-7_11

M3 - Book chapter

T3 - Lecture Notes in Computer Science

SP - 162

EP - 181

BT - HCI International 2023 – Late Breaking Papers

A2 - Degen, Helmut

A2 - Ntoa, Stavroula

A2 - Moallem, Abbas

PB - Springer

Y2 - 23 July 2023 through 28 July 2023

ER -

Mathiesen SO, Canossa A. If You Can’t Beat Them, Join Them: How Text-to-Image Tools Can Be Leveraged in the 3D Modelling Process. I Degen H, Ntoa S, Moallem A, red., HCI International 2023 – Late Breaking Papers: 25th International Conference on Human-Computer Interaction, HCII 2023, Copenhagen, Denmark, July 23–28, 2023, Proceedings, Part VI. Springer. 2023. s. 162-181. (Lecture Notes in Computer Science). doi: 10.1007/978-3-031-48057-7_11