Page 84 - IJAMD-1-2

International Journal of AI for
Materials and Design
AMTransformer for process dynamics

input data. Ko et al. utilized convolutional long short-term processes are characterized by their complex nature, with
9
memory (LSTM) to classify melt-pool anomalies, taking interrelated points, lines, and layers of material evolution
into account the spatiotemporal dependencies between during the processes at multiple scales. To accurately
melt pools. Zhang et al., also leveraged LSTM networks to capture AM dynamics, a comprehensive approach is
16
grasp the dependencies among input data characteristics, required – one that enhances the understanding of this
including process parameters and in situ MPM images. multi-scale complexity. A suitable modeling approach
Their model uses the LSTM networks to predict melt pool should not only extract the hidden features of various AM
size and subsequently utilizes a conditional generative states but also account for the underlying dependencies
adversarial network to generate predicted melt pool images. at multiple spatiotemporal scales. Understanding these
multi-scale dependencies is crucial, as they significantly
Recently, transformer methods have begun to emerge impact AM processes such as melt pool behaviors and,
in the field of AM. Fernandez-Zelaia et al. proposed a consequently, the overall quality of the final product.
26
video transformer framework that efficiently captures
both spatial and temporal patterns within a compact latent Furthermore, it is important to note that previous
space representation. This latent representation holds studies employing ML for AM dynamics have often
important physical information and is used to create a delineated their models within specific contexts, rendering
data-driven process-structure model. The study utilized them ad hoc and lacking adaptability to alterations in data
thermal simulation data from a reduced-order model to or the varieties of physical phenomena. Given the diversity
predict changes in spatial and temporal responses during of AM types, each with distinctive and different kinds of
manufacturing processes. Guirguis et al., used in situ data depending on its data acquisition systems, it is crucial
27
monitoring images that contain the dynamic changes in to consider a formal modeling approach when analyzing
the melt pools. They then leveraged spatiotemporal data the dynamics of AM.
to classify the process into different defect types and 4. Methods
conditions using video vision transformers. These studies
primarily focus on classifying and extracting features of This section introduces a novel ML method for learning
melt pools. AM dynamics: the AMTransformer. We begin by
presenting the dependencies between the physical entities
Transformer models have been proposed in these studies and their properties involved in AM dynamics. To achieve
to overcome the limitations of existing time-series neural this, we formulate the dynamical dependencies involved
networks. The pivotal concept of these models is the attention in AM phenomena using the concepts of state and rate
mechanism, providing a new paradigm for understanding properties. Following this, we detail the AMTransformer,
and processing sequences in ML models. The attention which enhances the ability to learn the dynamics of AM
28
mechanism allows an ML model to simultaneously attend processes and accurately predict their behaviors.
to all positions in the input data globally. This interaction
enables the model to assess the importance of all elements 4.1. Dynamical dependency formulations
in a sequence when processing a particular element. As a In modeling with the AMTransformer, we adapt the concept
result, even elements that are far apart in the sequence of dynamical dependency to represent the spatiotemporal
can effectively influence each other. In practical terms, the mechanisms of physical entities and properties involved in
attention mechanism allows the model to effectively handle AM dynamics. This adaptation provides a foundational
10
dependencies between elements, regardless of their distance representation of AM dynamics for ML modeling.
from each other. This approach contrasts with RNNs Dynamical dependencies illustrate the mechanisms of AM
and LSTMs, which process data sequentially, requiring and their impacts on changes in AM processes and parts. The
each time step to be analyzed only after all preceding properties of physical entities and the dependencies among
time steps have been processed. In the context of the AM them encapsulated within these dynamical dependencies,
dynamic systems, the ability of transformer models to represent the AM dynamics. The dynamical dependencies
examine interdependencies among sequential melt pools in AM dynamics are described as follows:
is fundamental for a comprehensive understanding of the • A physical entity, such as powdered metals, engages in
dynamics governing melt pool phenomena. an AM process, like the scanning of a laser. Figure 1

3. Problem statement illustrates various physical entities in an AM process.
• Each physical entity has a physical property, which is
Although existing ML studies in AM dynamics have shown divided into two sub-classes: state property and rate
promising performance, learning the fundamental dynamics property. A state property quantitatively characterizes
of AM processes remains a significant challenge. The AM the amount or momentum of a physical entity involved

Volume 1 Issue 2 (2024) 78 doi: 10.36922/ijamd.3919