Page 203 - IJOCTA-15-4

FastLoader: Leveraging large language models to accelerate cargo loading optimization with numerous
Comparison of algorithm runtime. By 4.4. Module ablation analysis
observing the algorithm runtime of evaluation un- Settings. In this comparison, we set up ablation
der four aircraft types, we find the following two experiments for the three modules of FastLoader
important observations: (1) compared with the to evaluate the impact of each module on this
heuristic algorithm, although FastLodaer loses
method.
some accuracy, it has a higher improvement in
Iteration analysis. As shown in Figure 5a-c,
algorithm runtime. This is because LLM effec-
we set different numbers of iterations in the eval-
tively reduces the search space, improves search
uation, and we have two key observations as fol-
efficiency, and reduces the time cost of the algo-
lows: (1) when FastLoader has a small number of
rithm through its own understanding of complex
iterations (e.g., 2 and 10 iterations), the balancing
constraints; (2) although the combinatorial opti-
results %MAC and %index still achieve high accu-
mization methods have the lowest algorithm run-
racy; (2) we particularly observed that when the
time, they cannot handle complex constraints, re-
number of iterations is low, the solution accuracy
sulting in the lowest solution accuracy.
of each large model is basically close. When the
Results. FastLoader maintains stable solu-
number of iterations exceeds 10, in the evaluation
tion accuracy in multiple LLMs and multiple air-
performance of %MAC, the results of Deepseek,
craft types, with an average deviation of no more
phi, and llama are better than the Qwen, which
than 1%. Compared with combinatorial optimiza-
shows these three LLMs have better capabilities
tion methods, FastLoader has an average improve-
in simplifying the load balancing search space un-
ment of 10% in accuracy. Compared with the
der complex constraints.
heuristic search methods, FastLoader has a loss
Token length analysis. As shown in Fig-
of no more than 1.5% in accuracy, but the algo- ure 5 d-f, we set different length of token in the
rithm runtime is reduced by an average of 90%, evaluation, and we have two key observations: (1)
which greatly reduces the time cost of calculating the accuracy of the solution drops significantly
the solution.
when the token is between 64 and 128 and tends
to converge in other token settings (256, 512); (2)
FastLoader has high accuracy and small differ-
4.3. Evaluation of runtime optimization ences under different token length settings. As a
result, FastLoader is robust to token length set-
Settings. In this comparison, we break down tings. The reason is that all methods almost over-
each step of the heuristic search and evaluate the lap when the token length is greater than 128 (the
algorithm runtime separately. We aim to ana- difference is less than 0.05% MAC). We conclude
lyze in depth which step of the heuristic search that whether it is Deepseek/phi based on instruc-
is optimized by Fastloader. Following the hyper- tion fine-tuning or the larger Qwen/Llama, as
parameter settings in the previous evaluation, we long as the length of the token exceeds 128, they
choose four heuristic search methods as baselines stably output the optimal solution in the same
to conduct the experiment. order of magnitude.
Comparison of runtime. Figures 3 and 4 Population size analysis. As shown in Fig-
show the algorithm runtime across four aircraft ure 5 g-i, we set different population sizes in the
types. We have two key observations as follows: evaluation, and we have the following three key
(1) the evolution phase accounts for 99% of the observations: (1) FastLoader achieves high accu-
heuristic search runtime. The main reason why racy in the results of %MAC and %index under
the heuristic algorithm consumes too much time different initial population size settings; (2) when
is that the search speed in the evolution phase is the population size exceeds 50, the solution ac-
slow, and it takes a long time to get an accurate curacy of all LLMs further improves, but when
solution; (2) by simplifying the search space with the population size exceeds 100, the accuracy of
LLM, FastLoader significantly reduces the run- Qwen3 decreases. This is because when Qwen3
time of the algorithm in the evolution phase. This generates the initial population, the similarity of
is because the proportion of feasible solutions in the solutions in the population is too high, caus-
the search space increases, while the overall size ing the result to fall into the local optimal solu-
of the search space decreases, so the runtime of tion. Under the same settings, other LLMs do not
solving the problem is effectively reduced. fall into the local optimal solution; (3) in terms
Results. By breaking down the heuristic of algorithm runtime, when the population size
search steps, we observe that FastLoader makes is set to 200, except for the time reduction of
effective optimizations to the runtime during the Qwen3, the algorithm runtime of the other LLMs
evolution phase, reducing the runtime by 90%. increases. However, the overall time to complete
745