Three dimensional (3D) virtual representations of the internal structure of textiles are of interest for a variety of purposes related to fashion, industry, education or other areas. The modeling of ancient weaving techniques is relevant to understand and preserve our heritage, both tangible and intangible. However, ancient techniques cannot be reproduced with standard approaches, which usually are aligned with the characteristics of modern, mechanical looms. The aim of this paper is to propose a mathematical modelling of ancient weaving techniques by means of matrices in order to be easily mapped to a virtual 3D representation. The work focuses on ancient silk textiles, ranging from the 15th to the 19th centuries. We also propose a computer vision-based strategy to extract relevant information from digital imagery, by considering different types of images (textiles, technical drawings and macro images). The work here presented has been carried out in the scope of the SILKNOW project, which has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No. 769504.Three dimensional (3D) virtual representations of the internal structure of textiles are of interest for a variety of purposes related to fashion, industry, education or other areas. The modeling of ancient weaving techniques is relevant to understand and preserve our heritage, both tangible and intangible. However, ancient techniques cannot be reproduced with standard approaches, which usually are aligned with the characteristics of modern, mechanical looms. The aim of this paper is to propose a mathematical modelling of ancient weaving techniques by means of matrices in order to be easily mapped to a virtual 3D representation. The work focuses on ancient silk textiles, ranging from the 15th to the 19th centuries. We also propose a computer vision-based strategy to extract relevant information from digital imagery, by considering different types of images (textiles, technical drawings and macro images). The work here presented has been carried out in the scope of the SILKNOW project, which has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No. 769504.