Historical weaving techniques have evolved in time and space giving as result more or less fabrics with different aesthetical characteristics. These techniques were transferred along the main silk production centers, thanks to the European Silk Road and creating a common European Frame on themes and techniques. These had made it complicated to determine whether a fabric corresponds to one century or another. Moreover, in order to understand their creation, it is necessary to determine the number of weaves and interlacements that each textile has, therefore, mathematical models can be extracted from these layers. In this sense, three dimensional (3D) virtual representations of the internal structure of textiles are of interest for a variety of purposes related to fashion, industry, education or other areas. The aim of this paper is to propose a mathematical modelling of historical weaving techniques by means of matrices in order to be easily mapped to a virtual 3D representation. The work focuses on historical silk textiles, ranging from the 15th to the 19th centuries. We also propose a computer vision-based strategy to extract relevant information from digital imagery, by considering different types of images (textiles, technical drawings and macro images). The work here presented has been carried out in the scope of the SILKNOW project, which has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No. 769504. The results shown in the paper are preliminary and will be improved in the scope of the project.