mechanical-engineering-skills

Tutorial: AMGPT (ask your AM library, get page-cited answers)

Skill: am-literature-rag · Paper: A. Chandrasekhar, J. Chan, F. Ogoke, O. Ajenifujah, A. Barati Farimani, AMGPT: A large language model for contextual querying in additive manufacturing, Additive Manufacturing Letters 11 (2024) 100232, doi:10.1016/j.addlet.2024.100232 · Code: BaratiLab/LLM_RAG

General chatbots answer metal-AM questions with outlines. AMGPT showed that retrieval-augmented generation (RAG) over a curated corpus gives specific, checkable answers. The setup was about 50 AM papers and textbooks, Mathpix PDF-to-TeX, LlamaIndex, all-mpnet-base-v2 embeddings and Llama2-7B. Expert graders found 80% of its answers factual and hallucination-free (GPT-4: 86.7%), and its answers were more specific, from a model with under 0.5% of GPT-4’s parameters.

In this skill the agent is the generator. amrag.py is the retriever, and every passage comes with a page-exact citation.

Use cases

Setup

npx skills add BaratiLab/mechanical-engineering-skills --skill am-literature-rag
pip install pypdf rank_bm25            # + sentence-transformers for the paper's dense retriever
cd skills/am-literature-rag

1. Build a library

We indexed the 13 Barati Farimani Lab papers used across this repo’s tutorials:

python scripts/amrag.py ingest ~/papers/*.pdf --index lib/
python scripts/amrag.py list --index lib/

The result is 13 documents and 695 chunks. Titles, DOIs and arXiv IDs come from PDF metadata and first-page text; reference lists are dropped. Need papers? reading-list --topic "melt pool" prints AMGPT’s curated corpus with DOIs.

2. Ask, read, answer only from the passages

python scripts/amrag.py query "How large are keyhole pores and what melt pool ratio indicates keyholing?" --index lib/ --k 5 --per-doc 2

Top passages:

  1. AdditiveLLM, p.1
  2. AdditiveLLM, p.1 (W/D below a threshold approximates keyhole porosity)
  3. MeltpoolNet, p.2 (high energy density leads to keyhole mode)
  4. ThermoPore, p.4 (pore thresholding)
  5. ThermoPore, p.3

The pore size sentence wasn’t in the top 5, so the agent runs a targeted follow-up:

python scripts/amrag.py query "average diameter of keyhole pores vapor column" --index lib/ --k 2

→ ThermoPore, p.4.

Answer written from those passages:

Keyholing is usually inferred from melt-pool geometry: a width-to-depth ratio below a threshold (1.5 in AdditiveLLM’s criteria) indicates likely keyhole porosity [AdditiveLLM: Large language models predict defects…, p.1]. It appears at high energy density, when the melt pool deepens beyond conduction mode [MeltpoolNet: Melt pool characteristic prediction in…, p.2]. Pores formed by the vapor column averaged about 47 µm in diameter in the work ThermoPore cites [ThermoPore: Predicting part porosity based on…, p.4].

Every sentence traces to a page. If nothing relevant comes back, the answer says the library doesn’t cover the question.

3. Check the retriever on your own questions

examples/questions_lab.json holds 12 questions with the file and a phrase the right passage must contain:

python scripts/amrag.py eval examples/questions_lab.json --index lib/ --k 3 --backend bm25
k Hit rate (BM25)
1 8 / 12
3 11 / 12
5 11 / 12

The miss asks for “unsafe” commands; the paper says “unintended shutdowns”. Lexical retrieval can’t bridge a paraphrase like that. The dense model (--backend dense, AMGPT’s all-mpnet-base-v2) or the default hybrid can.

Tips from the paper

Cite: Chandrasekhar et al., Addit. Manuf. Lett. 11 (2024) 100232.