Q-Slim: Entanglement-Guided Quantum Compression for Edge Small Language Models | IJLTEMAS
00
Days
00
Hrs
00
Min
00
Sec
Submit Your Paper

Q-Slim: Entanglement-Guided Quantum Compression for Edge Small Language Models

Authors

Dr. Karthiga I.

Department of Computer Science and Engineering, B.S. Abdur Rahman Crescent Institute of Science & Technology (India)

Lakshmi Narayanan P.

Department of Computer Science and Engineering, B.S. Abdur Rahman Crescent Institute of Science & Technology (India)

Mohamed Hamdhan S.

Department of Computer Science and Engineering, B.S. Abdur Rahman Crescent Institute of Science & Technology (India)

Article Information

DOI: 10.51583/IJLTEMAS.2026.151000124

Subject Category: Quantum AI

Volume/Issue: 15/10 | Page No: 1542-1552

Publication Timeline

Submitted: 2026-10-27

Accepted: 2026-11-01

Published: 2026-11-19

Abstract

Deploying Small Language Models (SLMs) with 0.5 billion to 1.5 billion parameters directly on consumer edge hardware preserves data privacy and removes cloud network latency, yet practical execution remains constrained by static Dynamic Random-Access Memory (DRAM) capacity and memory-bandwidth bottlenecks. The objective of this study is to compress pretrained SLMs below 4-bit average precision without suffering the severe multi-step reasoning degradation observed in uniform Post-Training Quantization (PTQ) or isolated block-diagonal Hessian approximations. To achieve this objective, we propose Q-Slim, a hybrid quantum-classical post-training compression framework combining tensor-network structural profiling with variational quantum combinatorial optimization. First, Q-Slim folds transformer Self-Attention and Multilayer Perceptron (MLP) weight matrices into higher-order tensors and decomposes them into canonical Matrix Product State (MPS) tensor trains to compute bond-wise von Neumann entanglement entropy as a calibration-free indicator of layer information capacity. Second, we derive an unconstrained two-qubit binary-radix variable encoding per layer group that maps discrete bit-width ({2, 3, 4, 8}-bit) and bond-truncation choices onto an 8-to-10 qubit Ising spin Hamiltonian using an exact single-layer distortion polynomial and a two-body quadratic coupling and memory surrogate, followed by deterministic greedy feasibility repair. Solving overlapping block windows with the Quantum Approximate Optimization Algorithm (QAOA) and unitary-folding Richardson Zero-Noise Extrapolation (ZNE) on the Qiskit Aer noisy density-matrix simulator yields a 97.6% min-max normalized approximation ratio against exact exhaustive enumeration. Evaluated on Qwen2.5-1.5B and Llama-3.2-1B under strictly matched 72.0% and 78.4% memory reductions, Q-Slim achieves 9.21 and 10.92 WikiText-2 perplexity on Qwen2.5-1.5B, outperforming uniform 4-bit and 3-bit GPTQ and AWQ while delivering 13.4 to 14.5 tokens per second on a Raspberry Pi 5. In conclusion, entanglement-guided quantum combinatorial allocation provides a practical, reproducible pathway for extreme edge SLM deployment via an interactive FastAPI and React studio.

Keywords

Small Language Models, Post-Training Quantization, Matrix Product States, Quantum Approximate Optimization Algorithm, Zero-Noise Extrapolation.

Downloads

References

1. Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., ... & Qiu, T. (2024). Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. https://arxiv.org/abs/2412.15115 [Google Scholar] [Crossref]

2. Meta AI. (2024). Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. Hugging Face Model Card. https://huggingface.co/meta-llama/Llama-3.2-1B [Google Scholar] [Crossref]

3. Frantar, E., Ashkboos, S., Hoefler, T., & Alistarh, D. (2023). GPTQ: Accurate post-training quantization for generative pre-trained transformers. In Proceedings of the International Conference on Learning Representations (ICLR). https://arxiv.org/abs/2210.17323 [Google Scholar] [Crossref]

4. Lin, J., Tang, J., Tang, H., Yang, S., Chen, W. M., Wang, W. C., ... & Han, S. (2024). AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration. Proceedings of Machine Learning and Systems (MLSys), 6, 87–100. https://arxiv.org/abs/2306.00978 [Google Scholar] [Crossref]

5. Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., & Han, S. (2023). SmoothQuant: Accurate and efficient post-training quantization for large language models. In Proceedings of the International Conference on Machine Learning (ICML) (Vol. 202, pp. 38087–38099). PMLR. https://arxiv.org/abs/2211.10438 [Google Scholar] [Crossref]

6. Dong, Z., Yao, Z., Gholami, A., Mahoney, M. W., & Keutzer, K. (2020). HAWQ-V2: Hessian aware trace-weighted quantization of neural networks. Advances in Neural Information Processing Systems (NeurIPS), 33, 295–306. https://arxiv.org/abs/1911.03852 [Google Scholar] [Crossref]

7. Orús, R. (2014). A practical introduction to tensor networks: Matrix product states and projected entangled pair states. Annals of Physics, 349, 117–158. https://arxiv.org/abs/1306.2164 [Google Scholar] [Crossref]

8. Oseledets, I. V. (2011). Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5), 2295–2317. https://epubs.siam.org/doi/10.1137/090752286 [Google Scholar] [Crossref]

9. Lucas, A. (2014). Ising formulations of many NP problems. Frontiers in Physics, 2, 5. https://arxiv.org/abs/1302.5843 [Google Scholar] [Crossref]

10. Farhi, E., Goldstone, J., & Gutmann, S. (2014). A quantum approximate optimization algorithm. arXiv preprint arXiv:1411.4028. https://arxiv.org/abs/1411.4028 [Google Scholar] [Crossref]

11. Temme, K., Bravyi, S., & Gambetta, J. M. (2017). Error mitigation for short-depth quantum circuits. Physical Review Letters, 119(18), 180509. https://link.aps.org/doi/10.1103/PhysRevLett.119.180509 [Google Scholar] [Crossref]

12. Cai, Z., Babbush, R., Benjamin, S. C., Endo, S., Huggins, W. J., Li, Y., ... & O’Brien, T. E. (2023). Quantum error mitigation. Reviews of Modern Physics, 95(4), 045005. https://link.aps.org/doi/10.1103/RevModPhys.95.045005 [Google Scholar] [Crossref]

13. Javadi-Abhari, A., Treinish, M., Krsulich, K., Wood, C. J., Lishman, J., Gacon, J., ... & Gambetta, J. M. (2024). Quantum computing with Qiskit. arXiv preprint arXiv:2405.08810. https://arxiv.org/abs/2405.08810 [Google Scholar] [Crossref]

14. Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2017). Pointer sentinel mixture models. In Proceedings of the International Conference on Learning Representations (ICLR). https://arxiv.org/abs/1609.07843 [Google Scholar] [Crossref]

15. Novikov, A., Podoprikhin, D., Osokin, A., & Vetrov, D. P. (2015). Tensorizing neural networks. Advances in Neural Information Processing Systems (NeurIPS), 28, 442–450. https://arxiv.org/abs/1509.06569 [Google Scholar] [Crossref]

16. Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., & Tafjord, O. (2018). Think you have solved question answering? Try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457. https://arxiv.org/abs/1803.05457 [Google Scholar] [Crossref]

17. Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., & Choi, Y. (2019). HellaSwag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL) (pp. 4791–4800). https://arxiv.org/abs/1905.07830 [Google Scholar] [Crossref]

18. Sakaguchi, K., Le Bras, R., Bhagavatula, C., & Choi, Y. (2020). WinoGrande: An adversarial Winograd schema challenge at scale. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 8732–8740. https://arxiv.org/abs/1907.10641 [Google Scholar] [Crossref]

Metrics

Views & Downloads

Similar Articles

Engineering,
Management &
Applied Science
Journal
(IJLTEMAS)
Published by RSIS International
Tower B, Suncity Success Tower
Golf Course Ext Rd, Sector 65
Gurugram, Haryana 122101, India
© 2026 IJLTEMAS · RSIS International. ISSN 2278-2540.