<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN"
  "https://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink"
         xmlns:mml="http://www.w3.org/1998/Math/MathML"
         article-type="research-article"
         dtd-version="1.2">

  <!-- ============================================================ FRONT -->
  <front>
    <journal-meta>
      <journal-id journal-id-type="publisher-id">IJLTEMAS</journal-id>
      <journal-title-group>
        <journal-title>International Journal of Latest Technology in Engineering, Management &amp; Applied Science (IJLTEMAS)</journal-title>
        <abbrev-journal-title abbrev-type="publisher">IJLTEMAS</abbrev-journal-title>
      </journal-title-group>
      <issn pub-type="epub">2278-2540</issn>
      <publisher>
        <publisher-name>IJLTEMAS</publisher-name>
      </publisher>
    </journal-meta>

    <article-meta>
      <!-- IDs -->
      <article-id pub-id-type="publisher-id">174</article-id>
            <article-id pub-id-type="doi">10.51583/IJLTEMAS.2026.150700173</article-id>
      
      <!-- Categories -->
            <article-categories>
        <subj-group subj-group-type="heading">
          <subject>Education</subject>
        </subj-group>
      </article-categories>
      
      <!-- Title -->
      <title-group>
        <article-title>A Hybrid LLM Based Relationship Extraction Algorithm for Unstructured Data</article-title>
      </title-group>

      <!-- Authors -->
      <contrib-group>
                <contrib contrib-type="author">
                    <name>
            <surname>Kaur Walia</surname>
            <given-names>Sukhvinder</given-names>
          </name>
                              <aff>
            School of Computer Science &amp; IT, Devi Ahilya Vishwa Vidalaya, Indore, M.P. India                        <country>India</country>
                      </aff>
                    
        </contrib>
                <contrib contrib-type="author">
                    <name>
            <surname>Shraddha Masih</surname>
            <given-names>Dr.</given-names>
          </name>
                              <aff>
            School of Computer Science &amp; IT, Devi Ahilya Vishwa Vidalaya, Indore, M.P. India                        <country>India</country>
                      </aff>
                    
        </contrib>
                <contrib contrib-type="author">
                    <name>
            <surname>Ugrasen Suman</surname>
            <given-names>Dr.</given-names>
          </name>
                              <aff>
            School of Computer Science &amp; IT, Devi Ahilya Vishwa Vidalaya, Indore, M.P. India                        <country>India</country>
                      </aff>
                    
        </contrib>
              </contrib-group>

      <!-- Volume / Issue / Pages -->
            <volume>15</volume>
                  <issue>7</issue>
                        <fpage>2374</fpage>
            <lpage>2383</lpage>
            
      <!-- Dates -->
      <history>
                <date date-type="received">
          <day>10</day>
          <month>08</month>
          <year>2026</year>
        </date>
                        <date date-type="accepted">
          <day>15</day>
          <month>08</month>
          <year>2026</year>
        </date>
              </history>

            <pub-date pub-type="epub">
        <day>27</day>
        <month>08</month>
        <year>2026</year>
      </pub-date>
      
      <!-- DOI Self-URI -->
            <self-uri xlink:href="https://doi.org/10.51583/IJLTEMAS.2026.150700173"/>
      
      <!-- Keywords -->
            <kwd-group kwd-group-type="author">
                <kwd>crime</kwd>
                <kwd>relationships</kwd>
                <kwd>triplets</kwd>
                <kwd>LLM</kwd>
                <kwd>NER</kwd>
                <kwd>criminal justice</kwd>
              </kwd-group>
      
    </article-meta>
  </front>

  <!-- ============================================================ BODY (Abstract) -->
  <body>
        <sec>
      <title>Abstract</title>
      <p>Unstructured text data such as crime reports and witness statements often contain essential connections between crime entities such as suspects, weapons, locations, and crime types. However, it can be difficult to extract and analyze these linkages because they are often inserted within unstructured narratives. This paper presents a hybrid framework that combines crime-specific entity recognition with fine-tuned large language models (LLMs) to extract relationships from crime data. The pipeline initially employs a BERT-based model to identify crime-related entities, followed by a Tiny LLAMA and LoRA-optimized LLM to discern and structure relationships as subject–predicate–object triplets which helps in criminal analytics. Experiments conducted on selected crime dataset showed significant improvements in 85.8 % accuracy with F1-score, recall, and precision over baseline models, especially when handling complicated relationship types. The results show that transformer-based NER combined with targeted LLM fine-tuning offers an effective hybrid approach for extracting useful relations from unstructured crime texts, which is helpful to criminal justice analytics and law enforcement.</p>
    </sec>
      </body>

  <!-- ============================================================ BACK (References) -->
    <back>
    <ref-list>
      <title>References</title>
            <ref id="ref1">
        <label>1</label>
        <mixed-citation>Sboev, A.; Rybka, R.; Selivanov, A.; Moloshnikov, I.; Gryaznov, A.; Naumov, A.; Sboeva, S.; Rylkov, G.; Zakirova, S. Accuracy Analysis of the End-to-End Extraction of Related Named Entities from Russian Drug Review Texts by Modern Approaches Validated on English Biomedical Corpora. Mathematics 2023, 11, 354. [Google Scholar] [CrossRef]</mixed-citation>
      </ref>
            <ref id="ref2">
        <label>2</label>
        <mixed-citation>Lezama-Sánchez, A.L.; Tovar Vidal, M.; Reyes-Ortiz, J.A. An Approach Based on Semantic Relationship Embeddings for Text Classification. Mathematics 2022, 10, 4161. [Google Scholar] [CrossRef]</mixed-citation>
      </ref>
            <ref id="ref3">
        <label>3</label>
        <mixed-citation>Wang, H.; Zhang, F.; Xie, X.; Guo, M. DKN: Deep knowledge-aware network for news recommendation. In Proceedings of the 2018 World Wide Web Conference, Lyon, France, 23–27 April 2018; pp. 1835–1844. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref4">
        <label>4</label>
        <mixed-citation>Sun, Q.; Xu, T.; Zhang, K.; Huang, K.; Lv, L.; Li, X.; Zhang, T.; Dore-Natteh, D. Dual-Channel and Hierarchical Graph Convolutional Networks for document-level relation extraction. Expert Syst. Appl. 2022, 205, 117678. [Google Scholar] [CrossRef]</mixed-citation>
      </ref>
            <ref id="ref5">
        <label>5</label>
        <mixed-citation>Le, H.Q.; Can, D.C.; Collier, N. Exploiting document graphs for inter sentence relation extraction. J. Biomed. Semant. 2022, 13, 1–15. [Google Scholar] [CrossRef] [PubMed]</mixed-citation>
      </ref>
            <ref id="ref6">
        <label>6</label>
        <mixed-citation>Zhou, D.; Zhong, D.; He, Y. Biomedical relation extraction: From binary to complex. Comput. Math. Methods Med. 2014, 2014, 298473. [Google Scholar] [CrossRef] [Green Version]</mixed-citation>
      </ref>
            <ref id="ref7">
        <label>7</label>
        <mixed-citation>Luo, H., Xu, C., &amp; Chen, Y. (2024). CRE-LLM: A Domain-Specific Chinese Relation Extraction Framework with Fine-tuned Large Language Model. arXiv. https://arxiv.org/abs/2406.14745 arXiv.</mixed-citation>
      </ref>
            <ref id="ref8">
        <label>8</label>
        <mixed-citation>Lai, P.T.; Lu, Z. BERT-GT: Cross-sentence n-ary relation extraction with BERT and Graph Transformer. Bioinformatics 2020, 36, 5678–5685. [Google Scholar] [CrossRef]</mixed-citation>
      </ref>
            <ref id="ref9">
        <label>9</label>
        <mixed-citation>Bui, Q.C.; Sloot, P.M.; Van Mulligen, E.M.; Kors, J.A. A novel feature-based approach to extract drug–drug interactions from biomedical text. Bioinformatics 2014, 30, 3365–3371. [Google Scholar] [CrossRef] [PubMed] [Green Version]</mixed-citation>
      </ref>
            <ref id="ref10">
        <label>10</label>
        <mixed-citation>Surdeanu, M.; Tibshirani, J.; Nallapati, R.; Manning, C.D. Multi-instance multi-label learning for relation extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Jeju Island, Republic of Korea, 12–14 July 2012; pp. 455–465. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref11">
        <label>11</label>
        <mixed-citation>Zeng, D.; Liu, K.; Lai, S.; Zhou, G.; Zhao, J. Relation classification via convolutional deep neural network. In Proceedings of the COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, Dublin, Ireland, 23–29 August 2014; pp. 2335–2344. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref12">
        <label>12</label>
        <mixed-citation>Xiao, M.; Liu, C. Semantic relation classification via hierarchical recurrent neural network with attention. In Proceedings of the COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan, 11–16 December 2016; pp. 1254–1263. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref13">
        <label>13</label>
        <mixed-citation>Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA, 2–7 June 2019; Volume 1, pp. 4171–4186. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref14">
        <label>14</label>
        <mixed-citation>Sun, Y.; Wang, S.; Li, Y.; Feng, S.; Tian, H.; Wu, H.; Wang, H. Ernie 2.0: A continual pre-training framework for language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 7–12 February 2020; pp. 8968–8975. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref15">
        <label>15</label>
        <mixed-citation>Wang, Y.; Sun, Y.; Ma, Z.; Gao, L.; Xu, Y.; Wu, Y. A method of relation extraction using pre-training models. In Proceedings of the 2020 13th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China, 12–13 December 2020; pp. 176–179. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref16">
        <label>16</label>
        <mixed-citation>Wei, Q.; Ji, Z.; Si, Y.; Du, J.; Wang, J.; Tiryaki, F.; Wu, S.; Tao, C.; Roberts, K.; Xu, H. Relation extraction from clinical narratives using pre-trained language models. In Proceedings of the AMIA annual symposium proceedings. American Medical Informatics Association, Washington, DC, USA, 16–20 November 2019; Volume 2019, p. 1236. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref17">
        <label>17</label>
        <mixed-citation>Luciano Del Corro and Rainer Gemulla. Clausie: clause-based open information extraction. In Proceedings of the 22nd international conference on World Wide Web, pages 355–366. ACM, 2013.</mixed-citation>
      </ref>
            <ref id="ref18">
        <label>18</label>
        <mixed-citation>Han, X.; Gao, T.; Yao, Y.; Ye, D.; Liu, Z.; Sun, M. OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, Hong Kong, China, 3–9 November 2019; pp. 169–174. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref19">
        <label>19</label>
        <mixed-citation>Cho, C.; Choi, Y.S. Dependency tree positional encoding method for relation extraction. In Proceedings of the 36th Annual ACM Symposium on Applied Computing, Virtual, 22–26 March 2021; pp. 1012–1020. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref20">
        <label>20</label>
        <mixed-citation>Rink, B.; Harabagiu, S. Utd: Classifying semantic relations by combining lexical and semantic resources. In Proceedings of the 5th International Workshop on Semantic Evaluation, Los Angeles, CA, USA, 15–16 July 2010; pp. 256–259. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref21">
        <label>21</label>
        <mixed-citation>Zhao, S.; Grishman, R. Extracting relations with integrated information using kernel methods. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), Ann Arbor, MI, USA, 25–30 June 2005; pp. 419–426. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref22">
        <label>22</label>
        <mixed-citation>Kim, S.; Liu, H.; Yeganova, L.; Wilbur, W.J. Extracting drug–drug interactions from literature using a rich feature-based linear kernel approach. J. Biomed. Inform. 2015, 55, 23–30. [Google Scholar] [CrossRef] [Green Version]</mixed-citation>
      </ref>
            <ref id="ref23">
        <label>23</label>
        <mixed-citation>Choi, S.P. Extraction of protein–protein interactions (PPIs) from the literature by deep convolutional neural networks with various feature embeddings. J. Inf. Sci. 2018, 44, 60–73. [Google Scholar] [CrossRef] [Green Version]</mixed-citation>
      </ref>
            <ref id="ref24">
        <label>24</label>
        <mixed-citation>Liu, S.; Tang, B.; Chen, Q.; Wang, X. Drug-drug interaction extraction via convolutional neural networks. Comput. Math. Methods Med. 2016, 2016, 6918381. [Google Scholar] [CrossRef] [Green Version]</mixed-citation>
      </ref>
            <ref id="ref25">
        <label>25</label>
        <mixed-citation>Peng, Y.; Lu, Z. Deep learning for extracting protein-protein interactions from biomedical literature. In Proceedings of the BioNLP 2017, Vancouver, BC, Canada, 4 August 2017; pp. 29–38. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref26">
        <label>26</label>
        <mixed-citation>Zhou, H.; Ning, S.; Yang, Y.; Liu, Z.; Lang, C.; Lin, Y. Chemical-induced disease relation extraction with dependency information and prior knowledge. J. Biomed. Inform. 2018, 84, 171–178. [Google Scholar] [CrossRef] [PubMed]</mixed-citation>
      </ref>
            <ref id="ref27">
        <label>27</label>
        <mixed-citation>Xu, Y.; Jia, R.; Mou, L.; Li, G.; Chen, Y.; Lu, Y.; Jin, Z. Improved relation classification by deep recurrent neural networks with data augmentation. In Proceedings of the COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan, 11–16 December 2016; pp. 1461–1470. [Google Scholar]</mixed-citation>
      </ref>
            <ref id="ref28">
        <label>28</label>
        <mixed-citation>Li, F.; Zhang, M.; Fu, G.; Ji, D. A neural joint model for entity and relation extraction from biomedical text. BMC Bioinform. 2017, 18, 198. [Google Scholar] [CrossRef] [PubMed] [Green Version]</mixed-citation>
      </ref>
            <ref id="ref29">
        <label>29</label>
        <mixed-citation>Wang, W.; Yang, X.; Yang, C.; Guo, X.; Zhang, X.; Wu, C. Dependency-based long short term memory network for drug-drug interaction extraction. BMC Bioinform. 2017, 18, 99–109. [Google Scholar] [CrossRef] [PubMed] [Green Version]</mixed-citation>
      </ref>
          </ref-list>
  </back>
  
</article>
