Image-based impedance spectroscopy for printed electronics

Introduction

Impedance is of great importance for evaluating electronic^1,2, electrochemical³, and biological^4,5 systems. In particular, in electronics, resistance induces voltage drop and eventual loss of electrical energy dissipated as heat, which suggests undesirable interaction with the environment. Furthermore, capacitance may introduce an unwanted response lag in a circuit. The characterization of impedance is essentially investigated in industries such as energy modules^6,7, power management integrated circuits (PMIC)^8,9, and radio frequency identification (RFID) antennas^10,11 for impedance matching, and a fast and simple measurement method has been sought. Impedance spectroscopy (IS) has been applied as a method to analyze systems such as metal corrosion¹², batteries^3,13, fuel cells¹⁴, and biomaterials transport¹⁵. However, characterization of impedance requires cumbersome experiments and may result in bottlenecks in multiple applications. In particular, epidermal electrodes have also been extensively researched based on experiments and simulations to minimize the impedance and optimize the radio frequency (RF) properties on contacted skin¹⁶ and they have been applied to electrophysiology, including electroencephalograms (EEGs)¹⁷, electrocardiograms (ECGs)^18,19, and electromyograms (EMGs)^20,21.

At the same time, printed electronics have gained significant attention for its versatility and scalability in flexible and large-area electronics for mass production^22,23,24,25. For instance, a printed biosensor is one of the most promising tools for initial diagnoses and has the potential to significantly reduce the cost for healthcare services^24,26,27,28. Printed electronics can be readily fabricated by inkjet printing²⁹, aerosol jet printing²⁶, screen printing^27,28, and 3D printing²⁴ of functional materials. Nonetheless, the impedance of the printed electrochemical biosensor needs to be recursively measured to ensure that it is reliable and reproducible. In addition, it is not easy to analyze all the electronics in continuous roll-to-roll (R2R) processes in industries and may result in a compromise between defects and processing time. An unavoidable failure during the process may create an electronics malfunction, especially a false diagnosis. Furthermore, when skin electronics degrades over time³⁰, it is not easy to measure its impedance on skin to guarantee performance. The above issues demand a non-contact measurement method for impedance to achieve reliability and processability.

Recently, machine learning has emerged as a revolutionary tool for conventionally difficult problems for restrictions in physical realization, sensor placement, and accessibility in the working environment^22,23,25,31. In particular, the convolutional neural network (CNN)³² has been successfully applied to reveal information from images^33,34, in which physical phenomena can be quantified in pixelated datasets and the weights in filters can be statistically trained for prediction²⁵. When the CNN is trained using a proper model architecture and dataset, it is possible to extract features from images for classification and regression problems^35,36. In addition, physics-informed neural networks (PINNs) have been implemented by including physical loss terms³⁷, updating variables in equations³⁸, and driving or solving equations³⁹ to integrate physics into the neural networks.

We propose a CNN⁴⁰ based image regression model to predict the impedance of printed electronics by just looking at images whose reflections and absorbance should be dependent on the surface morphologies of the nanoparticle grains and pores. A microscopic and impedance dataset was gathered under the comprehensive conditions of printing and sintering of inkjet printed electrode lines. We suggest an equivalent circuit of the printed electrodes to fit the impedance into the circuit elements. To this end, the correlations between image and impedance were learned using a CNN model to quantitatively predict the impedance from the images.

The image data-driven model was able to learn information hidden in image pixels to reveal the innate relationship of the optics with electronics and materials. It could solve a variety of problems that experts have identified but could not easily solve with engineering, including but not limited to impedance spectroscopy of printed electronics during the manufacturing process. The methodology is generic and suggests a revolutionary impedance analysis method for complexly structured and degrading electronic devices, which are often not accessible with direct measurement methods.

Results

Image-based impedance spectroscopy

Silver nanoparticle (AgNP) ink (DGP-40LT, ANP, Republic of Korea) was printed and sintered, as schematically shown in Fig. 1a. The printing and sintering parameters for the comprehensive study of printed electronics are suggested in Methods, and 375 printed lines were printed and measured for IS. The inkjet-printed AgNP line was comprised of pores, which could be observed with scanning electron microscopy (SEM) in Fig. 1b. The equivalent circuit of the inkjet-printed silver lines was modeled as a simplified Randles circuit⁴¹, which can be inferred from its Nyquist plot in Fig. 1b. It was the most simple and relevant equivalent circuit for the nanoparticle printed electrode interface. R₀ stands for the integration of the infinitesimal cross-sections with only Ag presence, whereas R₁ and C₁ stand for the integration of Ag and pores, respectively, in the cross-sections of both presences^42,43,44. The portions where only Ag or both Ag and pores exist are colored in blue and red in Fig. 1b, respectively. The physical meaning of the resistance and capacitance is further examined in the following sections. For data fitting into the equivalent circuit, not only the Nyquist plot, but also the Bode plot was comprehensively considered in the frequency domain. Ten measurements of impedance at different frequencies were fitted following the numerical method⁴⁵. An exemplary Bode plot and Nyquist plot are illustrated in Fig. 1b. The R₀, R₁, and C₁ values from IS were used as the true labels for image regression. The CNN model was trained using optical microscopic images as the input data to predict each component of the equivalent circuit. The CNN model is schematically displayed in Fig. 1c and its detailed model structure is provided in Supplementary Fig. 1. The linear activation function was used in the output layer for the regression of the R₀, R₁, and C₁ values.

Convolutional neural network trained under different light conditions

A 50-layer ResNet was used as the backbone structure of the CNN model⁴⁰, as shown in Supplementary Fig. 1. Its residual skip connections mitigated the curse of dimensionality for the highly deep neural network. Transfer learning from ImageNet⁴⁶ made the training process efficient. Dropout was used to prevent overfitting for generalization of the model⁴⁷. A single image was used for multi-output predictions of R₀, R₁, and C₁, by utilizing linear activation functions at the output layer.

There were seven different sorts of input data under the different sources of illumination for optical microscopy. Light sources were from top-center, top-side, top-ring, bottom, top-center and bottom, top-side and bottom, and top-ring and bottom of the lines, as schematically illustrated in Fig. 2. A total of 375 lines were printed and their impedances were measured for IS. Drop distance (β) was defined as the ratio of the drop distance (d) to the printed ink drop diameter (D₀)⁴⁸, and varied to 0.2, 0.35, 0.5, 0.65, and 0.8. A line printed for β > 0.8 was disconnected as its impedance was too high to be measured. The number of lines printed in the x-axis (#) was 1, 2, 3, 4, and 5, with the same drop distance in the y-axis, β_x = β_y. The temperatures of the sintering (T) were 100, 150, 200, 250, and 300 °C to comprehensively study the relationship between the nanoparticle sintering conditions and the electric property of the printed circuit. Each of 125 different lines was printed three times to evaluate its reproducibility for impedance regression using an image. Each of 375 lines was investigated at its top, middle, and bottom portions for data augmentation. Eventually 7875 images were collected, and 1125 images were used to train each CNN model using different sources of light input, as shown in Fig. 2.

**Fig. 2: Different light conditions for optical microscopy to attain inkjet-printed line image dataset.**

In addition, 5-fold cross-validation was used to train each CNN model to reduce overfitting while maintaining prediction performance⁴⁹. In turn, 5-fold cross-validation was voting ensembled to their arithmetic mean to enhance its predictive performance⁵⁰. The mean absolute error (MAE) was used as the loss function for robust training in spite of outliers⁵¹, which was likely to happen considering the complex printing and sintering processes, e.g., the interactions between the solvent and nanoparticles, especially during the sintering process.

An Inkjet printed line under different light sources was also exemplified with a schematic illustration of different light sources in Fig. 2. The top-center light source was intended to capture reflective light, as shown in Fig. 2b. The top-ring light was to capture scattered light, as shown in Fig. 2d. The top-side light was for capturing reflective and scattered light in Fig. 2c. The bottom light was for transmitted light in Fig. 2e. The top-center and bottom, top-side and bottom, and top-ring and bottom lights were to see the combinations of the above illuminations as shown in Fig. 2f–h, respectively. More images for varied sintering temperatures can be found in Supplementary Figs. 2-8.

A 5-fold ensembled validation score was plotted for each light source in Fig. 3a. The first histogram of Fig. 3a is the summation of R₀, R₁, C₁ MAE values. The R₀, R₁, C₁ values were normalized by subtracting mean values and subsequently dividing standard deviation values following Eq. 1,

$$widehat{x}=frac{x-bar{x}}{{sigma }_{x}}$$

(1)

where x ∈ {R₀, R₁, C₁}, (hat{{boldsymbol{x}}}) is the normalized value, (bar{{boldsymbol{x}}}) is the mean, and σ_x is the standard deviation. The normalization was conducted to predict R₀, R₁, C₁ values with a single neural network in a sense that R₀, R₁, C₁ values had significantly different orders of magnitudes, as shown in Fig. 3b.

**Fig. 3: Regression results of each CNN model trained on each light source image dataset.**

It is noteworthy that all MAEs for R₀, R₁, and C₁ were minimized when the light source was top-ring. The distinctive feature of the top-ring illumination image was that it had the most apparent contrast of the pores in the printed line. The pores were formulated during the heat treatment of the printed AgNP ink, which involved solvent drying, grain growth, and probable Ag island formation at high temperatures⁵². We deduced that the light scattering that came from the pores of the sintered nanoparticles were meaningfully captured by the top-ring light source⁵³. Therefore, we could effectively predict not only the resistance, but also the capacitance, which we thought to be from the pores of the printed lines⁵⁴. The pore is a crucial factor in impedance, and can be considered as air on the printed line with lower electric conductivity and higher permittivity than Ag, which increases both the resistance and capacitance. It is remarkable that the impedance of the printed electronics could mainly dependent on its pores, not to its bulging, or unintentional irregularities on the printed lines^55,56.

Bottom illumination or a combination with other lighting sources created higher MAE values. For instance, in Fig. 3a, top-ring illumination for scattering resulted in less loss than top-ring and bottom for scattering with transmitted light. The illumination from the bottom lighting source is blocked by the printed electrode lines. Consequently, the microscopic image is shown as a black line, which effectively captures the printed contour. However, it could not capture the relative depth or the surface information effectively, which is considered to be important for the impedance characterization. Thus, in general, bottom illumination produced relatively worse performance of the CNN model than center, ring, and side illuminations which capture surface morphology information from light reflectance, scattering, and the combination of reflectance and scattering, respectively. When bottom illumination is combined with other lighting sources, it hinders the combinatorial illumination model’s performances by reducing the relative brightness contrast on the electrode lines which conceals the surface morphology-induced features. This result suggests that illumination should be selective so that it does not conceal information, i.e., distinctive contrast overshadowed by uniform brightness. This issue is apparent in the microscopic images in Fig. 2 by comparing top-ring with top-ring and bottom images. To this end, designing experiments and obtaining a proper dataset is critical in machine learning, especially for engineering problems on top of a well-structured model.

The predictions from top-ring model at the top, middle, and bottom portions of the line images were averaged and compared to their true values from IS fitting in Fig. 3b. Resistances were well predicted despite several outliers, which had unusually high resistance values. Capacitance was also able to be predicted in spite of the order of magnitude difference from resistance, which was achieved from the normalization of the labels before the training process.

Image regression for printed electronics impedance prediction

The impedance regression result from the CNN model trained on the top-ring image dataset is mapped in Fig. 4. It is more likely to have small resistances R₀ and R₁ when there is more volume of printed ink, i.e., a smaller drop distance (β) and larger number of printed lines (#). We inferred that the CNN model could learn and predict resistance from the x, y dimensions and the z intensity of an image. Moreover, capacitance C₁ was more likely to increase, when there was a larger amount of ink printed, i.e., a smaller drop distance (β) and a larger number of printed lines (#). We thought that the capacitance was from the pores of the inkjet printed line, and the CNN model could learn from a larger number of pores on an image to predict the larger capacitance of the printed line. For sintering temperature, 200 °C generally showed the minimum resistances compared to the other thermal heat treatment conditions. For temperatures under 200 °C, AgNP was not integrated enough to form optimal bulk material, whereas at over 200 °C, it merged too much and started to be disconnected, and in turn, increased resistance and capacitance⁵². More regression results from the other sets of printed lines but with the same printing parameters can be found in Supplementary Figs. 9 and 10, which showed the same trend and proved reproducibility.

**Fig. 4: Impedance regression from top-ring CNN model.**

The results from IS can be found in Supplementary Figs. 11-13. The general trends of R₀, R₁, and C₁ values from IS corresponded to those from CNN. R₀ values from IS and CNN were close with a small difference. However, for some combinations of drop distance (β) and printing number (#), the orders of magnitude of R₁ and C₁ values from IS were different from the values from CNN. From that perspective, solely depending on IS was not enough as an impedance characterization method. The CNN model was not only a non-contact and nondestructive measurement method but also a more accurate method.

Convolutional neural network for precise impedance spectroscopy

The CNN-based model can be used not only for image regression, but also for precise impedance characterizations. First of all, the same initial conditions were used for all the IS measurements which served as true labels for CNN regression. After training, the generated CNN model predicted values for the IS. The image-based IS was successfully applied as a non-contact impedance measurement process. On top of that, the CNN regression could also be utilized to obtain more precise characterization than with IS, suggesting optimal initial values surpassing expert insights for IS, which is extensively dependent on initial conditions for excellent convergence. Even though IIS, itself, is promising as a fast non-contact and nondestructive evaluation, IIS-IS had its own potential. When both human experts and the CNN suggested a good choice of initial values for impedance, both IS and IIS-IS were well converged to the measured data, as shown in Fig. 5a and Table 1. On the other hand, when the CNN suggested more sophisticated values for initial conditions, IIS-IS was better converged to raw measurement data than only IS, as shown in Fig. 5b and Table 1.

**Fig. 5: Image-based impedance spectroscopy (IIS) – impedance spectroscopy (IS) to improve the precision of IS.**

Table 1 R₀, R₁, and C₁ values from IS and IIS-IS in Fig. 5

Full size table

In Fig. 5c, root mean square error (RMSE) values were calculated to compare the performances of IS and IIS-IS. The IS is the fitting impedance magnitude and phase from 10 measurement points using numerical method⁴⁵. Thus, all experimental sets for the varied drop distance (β), printing number (#), and sintering temperature (T) were used to calculate the RMSE to represent the general performance of the IS method. To compare the performance of IIS-IS with that of IS in the same setting, equally, all data were utilized to calculate the RMSE. The top ring image-trained CNN, which had the least MAE in Fig. 3a, was used to find initial values for IIS-IS. The RMSE of IIS-IS was 0.156, which was about half of the RMSE of IS, 0.290. Therefore, the image-based predictions of impedance using the CNN could not only be used for a non-contact first screening process, but also for precise measurement that would outperform the IS using initial values from suggestions from human experts. Furthermore, this research can be extended to iteratively solve the numerical method for an electronic circuit differential equation and train the CNN model. Future research should include a more sophisticated PINN algorithm to develop more accurate equivalent circuits from image data by improving the numerical method to solve the RLC differential equations with more circuit components.

IIS is particularly more advantageous when applied to inkjet printing. Inkjet printing is an ambient-pressure process, which does not require a vacuum chamber, differently with conventional evaporation or sputtering processes. Furthermore, it is a non-contact, computationally programmable process which is easily scalable with multiple printing nozzles for large-area materials, electronics, and biomedical applications. However, local non-uniformity could be an issue for inkjet printing whose process involves local patterning and sequential sintering, which does not occur with traditional evaporation or sputtering of bulk material with sequential patterning. Therefore, it is more important to develop a convenient impedance measurement method for printed electronics, while certifying the electronics characteristics. However, the traditional method requires cumbersome process to install and measure the electronics on a probe station. Furthermore, the measurements on a probe station may damage the electrodes, physically and electronically, which will gradually change the impedance characteristics over repeated characterizations. Therefore, in this research, we propose image-based impedance spectroscopy as a novel non-contact measurement method. The R2R process can be combined to further facilitate productivity. When the image-based impedance spectroscopy is used in the R2R process, the impedance can be continuously characterized following the printing and sintering processes. It will not only enhance productivity with tested characteristics but also keep the electronics intact during the manufacturing and the characterization.

Discussion

This research is the first effort for the development of a CNN-based image regression model for impedance spectroscopy (IS), which provides critical evaluation criteria for a broad range of industries. First, we applied it to the investigation of inkjet-printed AgNP, which can be used to obtain a comprehensive dataset with varied experimental parameters. We used K-fold cross-validation and ensemble to train the model while preventing overfitting. CNN models were trained on the different light source image datasets to extract features from images which are dependent on the nanomaterial morphologies and found their correlations with the impedance. Top-ring illumination showed the least MAE to predict impedance from images. The pores on the inkjet-printed lines could be captured with the most recognizable contrasts under top-ring illumination, which was considered to be the source of the capacitance of the printed electronics. We comprehensively studied the printing and sintering conditions, including drop distance (β), line printing number (#), and sintering temperature (T). When the drop distance was small and the number of line printing was large, in turn, the amount of printed ink was large, the resistances R₀ and R₁ were small, and the capacitance C₁ was large. Sintering temperature at 200 °C yielded the minimum resistances of the printed line, which was the temperature when AgNP was not under- nor over-heated to form bulk material. Three inkjet printing datasets showed the same trend and proved reproducibility. The CNN model demonstrated that it can learn image features not only in the x, y dimensions, but also in the z intensity, to predict the correlated electrical property. Furthermore, the CNN model can be utilized to improve the precision of IS. The RMSE of IIS-IS was about half that of IS. This result suggests that it can be used not only for the non-contact and nondestructive evaluation of printed electronics, but also for precise characterization that surpasses IS based on the initial values suggested by human experts. Moreover, the IS image regressor can be adopted for multiple electronic and biomedical engineering applications as a revolutionary characterization method.

Methods

Inkjet Printing

AgNP ink (DGP 40LT-15C, ANP, Republic of Korea) was filled in a cartridge (Samba, FUJIFILM Dimatix, Inc., USA) and printed by an inkjet printer (Marvel engineering, Republic of Korea). The drop distance (β) was defined as Eq. 2 and varied to 0.2, 0.35, 0.5, 0.65, and 0.8.

$$beta =frac{d}{{D}_{0}}=frac{{Drop,distance}}{{Drop,diameter}}$$

(2)

A line was printed following the y-axis on a bare glass wafer (Eagle XG, Corning, USA). The substrate was sonicated respectively in acetone and isopropyl alcohol (IPA) for 15 min for cleaning before the printing process. The printing number (#) was defined as the number of lines printed in the x-axis and varied to 1, 2, 3, 4, and 5. The drop distance in the y-axis, β_y, was the same as the drop distance in the x-axis, β_x, for each printing number, β_x = β_y. Inkjet-printed lines were sintered at the temperature (T) from 100, 150, 200, 250 to 300 °C, on a hot plate for 30 min. An infrared thermometer was also used to check the temperature on the substrate. A total of 125 lines were printed three times yielding 375 lines in total.

Impedance Spectroscopy

The impedance of the printed and sintered lines was measured using an LCR meter (E4980AL, Keysight, USA), which was connected to the probe station (1160 Series, Signatone, USA). The impedance was measured at ten frequencies, 30, 100, 300, 1000, 3000, 10,000, 30,000, 100,000, 300,000, and 500,000 Hz, and fitted by the numerical method.

Optical Microscopy

Inkjet-printed and sintered lines were observed on an optical microscope (VHX-6000, KEYENCE, Japan). Each line was observed three times at its top, middle, and bottom following the line in the y-axis. Top-center, top-side, top-ring, bottom, top-center and bottom, top-side and bottom, and top-ring and bottom light sources were used in the microscope to investigate each line image. In the end, 7875 images were obtained as an inkjet-printed line image dataset. The image size was 1200×1600×3, which was the height, width, and channel, respectively.

Convolutional Neural Network

The input image was reduced to 600×800×3, which was a size that was not significantly detrimental to the model metrics. For each image, the pixel values were normalized by dividing by 255. A 50-layer ResNet was the backbone structure of the CNN model, whose weights were pre-trained on an ImageNet dataset. The dropout layer was added for generalization of the model. A linear activation function was used to predict each impedance component, R₀, R₁, and C₁. A 5-fold cross-validation was used and ensembled to train the CNN model. A 5-fold cross-validation was conducted to make a training and testing comparison feasible for different illumination dataset-models whose number of data points were relatively small compared to conventional computer vision deep learning problems. All data were split to 5 folds to train and test with 5 models. Each fold was a different 20% of data which served as a validation set for testing of each illumination-data trained model. The MAE was utilized as the loss function. Five mean absolute error (MAE) values were calculated from 5 different validation sets of 5 models, respectively. The 5 MAE values were averaged and displayed in Fig. 3a to compare the performances of 7 different illumination dataset-trained models. The 7 CNN models were trained for each light source image set: top-center, top-side, top-ring, bottom, top-center and bottom, top-side and bottom, and top-ring and bottom. By doing so, the performances of the 7 models could be tested and compared with a small amount of data.