SuperMRF: deep robust reconstruction for highly accelerated magnetic resonance fingerprinting

Introduction

Magnetic resonance fingerprinting (MRF) (1,2) is a data acquisition technique that employs variable acquisition parameters, such as varied radiofrequency excitations and delays, to generate unique magnetic resonance (MR) signal evolutions (so-called “fingerprints”) for different tissues. Quantitative maps of tissue parameters, such as T1 and T2 relaxation times, are obtained by comparing the acquired signal evolution with precalculated fingerprints of candidate parameters in a dictionary using pattern matching. This approach enables multiple tissue parameter maps to be obtained simultaneously, which significantly reduces total scan time as compared to conventional, single-parameter quantitative MR imaging (MRI) techniques.

To enable accurate pattern matching for different tissues, the MRF signal evolution is typically acquired over a long period of time, which results in a long scan time, albeit faster than that of conventional methods. Built on the dictionary-based, inner product pattern matching approach, advanced iterative reconstruction algorithms have been developed to enable MRF at higher acceleration factors and reduced acquisition times while preserving image quality and quantification accuracy and precision. For example, iterative multiscale reconstruction has enabled a 3× reduction in acquisition time (3). Recently, low rank methods have become a “state-of-the-art” approach in MRF reconstruction, taking advantage of redundancy in the time domain via temporal subspace constraints and enforcing sparsity in the spatial domain with a compressed sensing type of iterative reconstruction (4-12). Low rank MRF reconstruction can decrease the acquisition time while still providing improved accuracy and precision as compared to inner product-based pattern matching (4). Beyond leveraging the low rank properties of the temporal subspace, spatial low rank reconstruction has also been demonstrated to improve reconstruction quality. A low rank tensor reconstruction (13) has been shown to yield lower T1 and T2 estimation error as compared to an exclusively temporal low rank reconstruction (4). Local image patches have been shown to exhibit even lower rankness than the full image space, enabling greater improvements in reconstruction quality, particularly when combined with a sparsity regularization such as the wavelet transform (6). However, a limitation of such reconstruction techniques is their computational burden, as multiple iterations between the image domain and k-space sampled data are required. Consequently, such iterative reconstructions are generally performed offline after the MRI exam is finished, which is problematic if the acquired data are suboptimal due to artifacts or inadequate prescribed geometry. There is a need for rapid, robust MRF reconstruction methods that can generate tissue parameter maps quickly, enabling inline reconstruction at the time of the MRI exam.

Current pattern-matching-based MRF reconstruction methods must contend with memory and computational challenges in order to yield accurate and reliable tissue parameter maps. To mitigate quantization effects in reconstructed tissue parameter maps, the MRF dictionary can be computed at a high resolution across the range of anticipated tissue values. While this approach works well at a low number of tissue parameters, the dictionary size can become prohibitively large as the number of parameters increases. For example, a 4-parameter dictionary (e.g., T1, T2, T1rho, and B1) with 1,000 time frames and 1,000 unique values for each parameter would require the generation of a dictionary array with the number of entries on the order of 10¹⁵ or 8 petabytes at double precision with 8-bytes per entry. Such an increase in memory, data storage, and computation hinders the clinical adoption of MRF methods as well as innovative, high-dimensional applications of MRF. The current solution of reducing the dictionary density via sub-sampling is insufficient because quantization effects degrade the accuracy of MRF reconstruction even under ideal experimental conditions. An alternative solution is to compress the dictionary using singular value decomposition (SVD) in the temporal dimension (14). Such temporal compression can significantly reduce the number of time points while preserving information relevant to tissue parameters, thus enabling low rank reconstruction methods as described previously. While temporal compression reduces dictionary size in the time domain, the number of tissue parameter combinations will still produce a dictionary with a large number of entries. Methods for computing the temporal compression of such large dictionaries have been enabled by memory efficient methods such as the randomized SVD (8), however, storing the temporally compressed dictionary in memory may remain impractical for more than 3 or 4 parameters given the large number of parameter combinations. Additionally, the computational burden for pattern matching in the low rank subspace may remain prohibitively high for more than 3 or 4 parameters. For these reasons related to computational burden, there is a need for rapid and robust MRF reconstruction methods that may be addressed by deep learning approaches.

Deep learning algorithms, particularly neural network based techniques, have been widely reported for MRI reconstruction problems and may enable computationally efficient, quantitatively accurate MRF reconstruction even under highly accelerated or noisy conditions (15-21). With respect to MRF, a deep learning method named deep reconstruction network (DRONE) was introduced to overcome the large dictionary size constraint by using a neural network (16). However, spatial correlation among neighboring pixels is not considered in DRONE, which might limit the method’s performance when considering noise or spatially coherent undersampling artifacts that occur as part of MRF acquisitions. Considering this issue, a recent deep learning framework adds the U-net (19) as a second module for spatially-constrained quantification (20). However, such a two-step method may be inefficient because a fully connected network (for feature extraction in time) and p U-nets (p being the number of tissue parameters of interest) need to be trained separately (20). In this paper, we propose a physical model-guided deep learning framework named “SuperMRF”, where the neural component is composed of a 3D convolutional neural network (CNN)-based deep-learning method that exploits both two-dimensional (2D) spatial and one-dimensional (1D) temporal information to directly map the MRF signal to the underlying tissue parameters. The CNN-based component used in SuperMRF is similar to SuperMAP and SuperDTI methods, which were recently demonstrated to be effective for parameter mapping (22,23). The proposed network builds upon those previously reported methods. Specifically, an additional loss term that minimizes the difference between the signal evolution of reconstruction and its corresponding fingerprint is introduced to incorporate the physical model based on Bloch simulation, which is unique to MRF. The proposed network also incorporates a computationally efficient linear temporal compression. The novelty of the approach is in the combination of these loss terms, temporal compression, and patch-based training that are tailored to address specific challenges in MRF reconstruction that include undersampling artifacts, complicated spatiotemporal signal patterns, low SNR, and the complicated loss terms in reconstruction networks. We investigate the use of SuperMRF for additional acceleration of MRF scanning in both the number of time frames as well as k-space noise robustness without compromising accuracy. Numerical simulations based on reference data from human subjects are used for well-controlled experiments with known ground truth. After training the model on numerical simulations, the SuperMRF reconstruction is evaluated in a prospective real-world evaluation of human subject MRF scans.

Methods

MRF sequence

The MRF sequence uses a 3D Cartesian trajectory with linear readout in k_x and a variable density circular Cartesian undersampling pattern in k_y-k_z (24). The sequence structure used for simulation has been described previously (25) and was designed as follows with the aim of quantifying T1 and T2 in cartilage and muscle. Cartesian k-space sampling was selected as it is known to be more robust to system imperfections and would not exhibit chemical shift displacements from fatty tissues in more than one direction, as has been observed in non-Cartesian implementations (26). The MRF acquisition consists of 1,000 time frames that are acquired using fast imaging with steady-state free precession, with 4π spoiler gradient, and with each frame acquired at a specified acceleration factor (AF). Multiple readouts at different k_y-k_z sample points are achieved by repeating the 1,000 frame acquisition after a period of time that is sufficient for longitudinal magnetization recovery (4 seconds assumed in this work). The acquisition matrix size is 96×96×10 with a resolution of 1.5×1.5×10 mm³, corresponding to a 140-mm field of view and 100 mm coverage in the slice direction. For the MRF acquisition itself, variable flip angles between 5 to 15 degrees, constant repetition time (TR) of 5 ms, and constant echo time (TE) of 2.5 ms, are used. Magnetization preparation pulses are employed to increase sensitivity to T1 and T2 values: an inversion pulse is applied prior to frame 1 with an inversion time of 21 ms, and T2 preparation pulses are applied prior to frames 101, 201, 301, 401, 501, 601, 701, 801, and 901 with echo times of 20, 40, 60, 80, 20, 40, 60, 80, and 20 ms, respectively. Also evaluated in this work were shortened sequence variants with 500 frames and 200 frames, wherein the number of preparation pulses was reduced while retaining the original scheduling (e.g., inversion and T2 preparation pulses at the same frame index and of the same preparation time design). An additional “long” sequence variant of 2,000 frames was included for comparison purposes where the same preparation schedule as the 1,000 frame sequence was used but repeated immediately after frame 1,000 (e.g., frames 1,001 to 2,000 were the same preparation and flip angle schedule as frames 1 to 1,000). A summary of MRF sequence variants is shown in Table 1, as well as a breakdown of data acquisition time, magnetization recovery time, and total scan time.

Conventional MRF reconstruction

Raw data are fully sampled in k-space for a single-channel receive system and then used to produce T1, T2, and M0 maps with the use of a dictionary and inner product-based pattern matching. Undersampled MRF frames are reconstructed using a 3D inverse fast Fourier Transform. A Bloch simulator is used to compute signal evolutions for unique combinations of T1 and T2 for the MRF sequence (1). Inner product pattern matching is performed on a voxel-wise basis between normalized measured signal evolutions from the undersampled images and normalized signal evolutions from the dictionary. T1 and T2 maps are generated from the combinations corresponding to the largest inner product values. M0 values are determined by the scaling between the original measured signal and the normalized best-matched signal.

Low rank MRF reconstruction

An iterative, sparsity, and local low rank reconstruction was used to serve as a comparison with SuperMRF (6). Briefly, the reconstruction algorithm computes temporally compressed images that exhibit data consistency as well as spatial local low-rankness and sparsity. Temporal SVD compression based on the MRF dictionary as previously described (14) was used to obtain the temporally low rank (singular value) images and the corresponding temporal compression matrix, U. Local 3D image patches were extracted from singular images and used for local low rank regularization. Wavelet regularization in the spatial domain was employed to further suppress residual noise and undersampling artifacts in the singular images. The reconstruction of singular images, x^, is posed as the following optimization problem (6),

x^=argxmin12‖MTUx−y‖22+λb‖Rbx‖*+λW‖Wx‖1[1]

where T is the 3D spatial Fourier Transform operator, M is a k-space sampling mask, y is the set of k-space MRF data measurements, R_b is the local Casorati reshaping operator for local neighborhood b chosen for rank minimization, ‖⋅‖∗ is the nuclear norm, λ_bis a parameter for local low rank minimization, W is the wavelet transform, ‖⋅‖1 is the l₁-norm, and λ_W is a parameter for wavelet regularization strength. The optimization is carried out using the alternating direction method of multipliers (ADMM) (6).

MRF digital phantom

Digital phantoms based on in vivo data from volunteers were used for the evaluation of reconstruction techniques. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Cleveland Clinic Institutional Review Board (Cleveland Clinic IRB number 17-1351, Federal Wide Assurance number FWA00005367) and written informed consent was taken from all individual participants. Volunteers included males and females and age range 22 to 34 years. Scans were performed on a 3T MRI scanner (Prisma, Siemens) with a single-channel transmit, 15-channel receive knee coil [Flare, Quality Electrodynamics (QED)]. The in vivo data consist of T1 and M0 maps obtained from 2D multislice phase-sensitive inversion recovery and T2 maps obtained from 3D Magnetization-prepared Angle-modulated Partitioned k-space Spoiled gradient echo Snapshots (3D MAPSS). Sequence parameters are shown in Table 2. Quantitative tissue parameter maps were used as input reference values for the generation of MRF data. The approach taken in these simulation experiments was to generate simulation data with exact corresponding ground truth T1 and T2 maps where errors would be exclusively due to limitations in spatial and contrast weight encoding. The simulation steps are as follows:

• First, the tissue parameter maps were tri-linearly interpolated to the matrix size of the MRF acquisition;
• Second, the reference T1 and T2 values were rounded to the nearest MRF dictionary entry;
• Third, the time series of fully sampled image volumes for each MRF frame was generated by placing the appropriate dictionary signal evolution in each voxel and scaling it by the reference M0 map;
• Fourth, the fully-sampled reference MRF data were transformed to k-space using a 3D fast Fourier Transform at each MRF frame and then the k_x-k_y-k_z undersampling mask was applied;
• Last, these undersampled k-space data were then used as inputs to the conventional and low rank MRF reconstruction methods. Undersampled images obtained after performing a 3D inverse fast Fourier Transform were used as inputs to the SuperMRF deep learning model reconstruction.

Digital phantom simulation and reconstruction was performed on a system with 64 GB of random access memory (RAM) and a 7980XE (Intel) central processing unit (CPU) (18 cores and 36 threads). The iterative low rank reconstruction is handled by the CPU with multi-threading to parallelize the computations. Simulations and reconstructions were implemented in MATLAB (Mathworks).

SuperMRF architecture

SuperMRF is a synergistic physical-neural framework. The neural network is consisted of a 10-layer encoder-decoder shaped network with skip connections. As shown in Figure 1, the neural network is guided by the physical model, namely the Bloch simulation. The hyperparameters of the network are also illustrated in Figure 1. Residual learning (27) with patch-wise training/padding (28,29) is used to guarantee the model robustness. The reconstruction is built using the MATLAB Caffe framework for Windows. The following sections will describe the details of the algorithm.

Figure 1 Flowchart of SuperMRF. Three losses are used during training: Loss1 for quantitative maps, Loss2 for de-aliasing, and Loss3 for consistency. Within a block, n256k3s1p(p’)1 states 256 filters with a kernel 3, stride 1, and padding (p’ for truncation) of 1. CONV, convolution; MR, magnetic resonance; MRF, magnetic resonance fingerprinting; RELU, rectified linear unit; SVD, singular value decomposition.

Loss terms for training

Three loss terms are chosen for SuperMRF training, as illustrated in Figure 1. Similar to the framework we previously proposed for diffusion tensor imaging (23), the traditional loss term (Loss1) guarantees the calculated parametric maps through the input 3D patches x_i are consistent with the target maps y_i. The T1, T2 and M0 maps were normalized to 0 and 1, according to their respective maximum values. Additional loss terms are proposed and included as part of SuperMRF. The second loss (Loss2) enforces de-aliasing by the network, which uses information across image patches, where image patches may contain identifiable aliasing characteristics in addition to the desired signal content. Bloch simulation is used to generate fully-sampled weighted images based on the network-estimated parameter maps, which are then compared to the reference fully-sampled parameter-weighed images. The third loss (Loss3) enforces data-consistency by applying the k-space undersampling pattern to the inferenced fully-sampled weighted images described as part of Loss2. Specifically, the fully-sampled weighted images are Fourier transformed to k-space, multiplied elementwise by the undersampling mask (30), and then transformed back to the image domain. Loss3 is computed by comparing these estimated undersampled images to the reference undersampled images.

SuperMRF training and reconstruction do not require an MRF dictionary, but a pre-computed dictionary may be used to expedite computations by avoiding many Bloch simulations as well as by providing temporal compression via singular vectors computed from the dictionary. In general, any continuous, real-valued tissue parameter combination can be used to embed a physics model in the network training process with the Bloch simulation. The information is utilized to train the network parameters by minimizing the loss function between the prediction from the input patch x^tand the corresponding ground truth map y^tfor all training samples t=1,…,n:

L(Θ)=Loss1(Θ)+λ2Loss2(Θ)+λ3Loss3(Θ)[2]

Loss1(Θ)=1n∑t=1n‖F(U(xt);Θ)−yt‖2[3]

Loss2(Θ)=1n∑t=1n‖U(B(F(U(xt);Θ)))−U(B(yt))‖2[4]

Loss3(Θ)=1n∑t=1n‖U(S(B(F(U(xt);Θ))))−U(S(B(yt)))‖2[5]

Where λ_iis a hyperparameter for weighting of each loss function; F represents the operation performed by the neural network; Θ denotes the network parameters to be learned during training; U is an optional temporal compression operation; B denotes the process of conversion of continuous, real-valued tissue parameters to MRF signals based on Bloch simulation; and S denotes the entire operation of putting patches into an image, undersampling the k-space, and extracting the patch from the undersampled image. In order to reduce computation time during training, we replaced the Bloch simulation with MRF signals from the pre-computed dictionary of quantized tissue parameter combinations after rounding to the nearest dictionary entry. Note that the tissue parameter maps generated from F(U(x^t);Θ) are not constrained to dictionary entries even with this substitution. The weights λ_i are determined by the size of the patch and the overlapping degree between patches. Since Loss1 and Loss2 are calculated patch-wise and Loss3 is based on the entire image, λ_i should be selected to ensure that all three losses are of comparable scale. The weight λ₃ is determined by the actual patch size and stride. For a single image of size 96×96, patch size 21, and 95% overlapping rate (stride of 1), there will be approximately 5.5k patches, so λ₃ is approximately 0.00018 (1:5,500).

Input and output

During training, both the input (temporal compressed images), dictionary of MRF signal time courses, and target output (ground truth T1, T2, M0) are provided to the network.

The proposed SuperMRF network can operate using all undersampled weighted images from an MRF acquisition as inputs, as well as using a reduced number of temporally compressed images. In this work, we use a temporal compression based on the SVD of the pre-computed MRF dictionary to compute singular vectors that efficiently reduce the number of input images from hundreds of individual undersampled MRF images down to a few ’singular images’ while retaining relevant MRF signal characteristics (14). This temporal compression can be regarded as a feature extraction procedure without the fully connected layers, as used in Fang et al. (20). Temporal compression is performed using the dictionary rather than measured signals in order to preserve information as well as ensure consistency of temporal compression between datasets. Note that deployment of the trained reconstruction network does not require the inclusion of the dictionary, in contrast to low rank reconstruction methods. Reduction of the number of input images reduces both the computational burden as well as the overfitting risks. For this work, three singular images from the temporally compressed image data are used as the input to generate the T1/T2/M0 maps as the SuperMRF output. The use of three singular images in this work corresponds to approximately 95% of the energy from the dictionary compression. In the training stage, the parameters Θ are learned and updated. In the testing stage, the acquired MRF data x are fed into the network F(U(x);Θ) for estimation of the desired maps.

Patch selection

Instead of using the entire image or single pixels, we convert the image into 2D overlapping patches (21×21 matrix, stride of 1, overlapping rate of 95%). Patch-based training can save memory and produce sufficient training data to prevent overfitting compared to using the entire image (28,29). On the other hand, the model from pixel-wise learning is not robust in practical situations where noise occurs. A sufficiently large patch size ensures learning of the spatial correlations for enhanced robustness to the aliasing artifacts. In this study, there are approximately 200,000 training patches when only 4 subjects are selected for training. The patch size is optimized empirically to balance the tradeoff between memory usage and data correlation. For different scanners and settings, the choice of patch size and the overlapping rate can be determined by the size of the original image and other parameters, which makes the network more generalizable.

Network training

Data from four subjects were used for training and validation and one was used for testing (31). The testing dataset was strictly excluded from the training stage.

Experiments were conducted on two NVIDIA-P6000 (24GB) graphical processing units (GPUs). During the first 80 epochs, the initial learning rate was set to 0.0001 with 0.9 momentum and a decay of 0.0001. The learning ratio was later reduced to 0.00002 for the last 20 epochs. The choice of learning rate values was made based on previous reports with similar rates used for image reconstruction tasks (23,32). Adaptive moment estimation (ADAM) was used as the optimization algorithm (33,34).

Prospective real-world evaluation

A prospective, real-world evaluation was performed on four volunteer subjects scanned on the same 3T scanner and coil described in section “MRF digital phantom”. The subjects included male and female volunteers and age range 24 to 79 years. Three of the subjects had no known knee pathology, whereas one subject had severe knee-related symptoms with a total knee arthroplasty planned within six months. The acquired data were reconstructed to obtain T1 and T2 maps using MRF, low rank MRF (LR-MRF), and SuperMRF. The SuperMRF reconstruction used the same model obtained from digital phantom experiments without any additional training.

Evaluation metrics

For the simulation datasets with known ground truth, normalized mean squared error (NMSE), peak signal to noise ratio (PSNR), and structural similarity index (SSIM) (35) were chosen for the quantification of the performance for full 3D image reconstruction with different methods. NMSEs were calculated for knee cartilage and skeletal muscle to estimate the errors of the estimated T1/T2 relaxation times. Cartilage and skeletal muscle were manually segmented using the M0 maps from the inversion recovery dataset. Segmentations were subsequently resampled via nearest-neighbor interpolation to the same size as the MRF and reference maps for NMSE calculation.

For the prospective real-world evaluation, large regions of interest (ROIs) were manually defined within four tissue regions for each subject (femoral cartilage, tibial cartilage, patellar cartilage, muscle). Mean T1 and T2 values within ROIs were recorded. The same ROI was used for each tissue region and reconstruction method (MRF, LR-MRF, SuperMRF) for direct comparison of reconstruction methods.

Comparison

The previously described state-of-the-art low rank method was used to estimate the T1 and T2 maps with the same acquisitions as used for SuperMRF evaluation. The same number of singular values were retained (three) for comparison between LR-MRF and SuperMRF.

To evaluate the robustness to noise, we added complex-valued Gaussian white noise to the sampled k-space data for the simulated MRF acquisitions to generate desired SNR levels. The signal of a dataset was defined as the mean absolute value across k-space measurements, ∑m|ym|M, where m is the index of a k-space measurement and M is the number of k-space measurements. The noise standard deviation that was added for a given dataset, σ, was proportional to the signal for that dataset,

σ=α∑m|ym|M[6]

where α is the proportionality constant that determines the noise level and the corresponding SNR level, i.e., SNR =1/α. The proportionality constant ranged from 0.02 to 0.10, giving SNR levels of SNR =50 to SNR =10. The same noisy k-space data were reconstructed with the low rank MRF reconstruction and SuperMRF reconstruction, and the NMSE and PSNR were evaluated with respect to reference T1, T2, and M0 maps.

To evaluate the quantification in prospective, real-world in vivo datasets, mean and standard deviation of T1 and T2 values were compared between reconstruction methods for each tissue region and overall. The overall T1 and T2 values from reconstruction methods were also compared by using paired t-tests (MRF vs. LR-MRF, MRF vs. SuperMRF, LR-MRF vs. SuperMRF). Differences were also evaluated by subtracting LR-MRF from MRF and SuperMRF, using mean absolute difference (in ms) and mean relative difference (in percentage) as metrics for comparison.

Results

Frame reducing test

Figures 2,3 show conventional MRF, low rank MRF (LR-MRF) and SuperMRF performance for T1 and T2 mapping with the number of MRF frames reduced from 1,000 to 50. Tissue property maps reconstructed by LR-MRF and SuperMRF were observed to be less sensitive to frame reduction and exhibited good reconstruction quality when frames were reduced by 80%. Whereas Conventional MRF showed noticeable degradation in T1 and T2 maps at any reduction below 1,000 frames, LR-MRF and SuperMRF were generally comparable between 1,000 and 200 frames. In the “ultra-short” acquisition of only 50 frames, SuperMRF produced T1 and T2 maps with quality comparable to the 1,000 frame Conventional MRF maps, whereas LR-MRF produced T1 over-estimation that is particularly evident in the soleus and gastrocnemius muscles. Overall quality metrics indicated the preferable performance of SuperMRF over the range of frame reduction conditions (Table 3). M0 maps were not analyzed in this work but are shown in the Supplementary file (Appendix 1).

Figure 2 Traditional MRF, LR-MRF and SuperMRF for T1 mapping with the number of frames reducing from 1,000 to 50 and a fixed k-space AF of 15. Units are in ms. Blue arrows indicate regions of note in the cartilage that LR-MRF appreciably differed from reference T1 values. Green arrows indicate regions of muscle (soleus and gastrocnemius) that LR-MRF exhibited appreciable elevation in T1 values as compared to reference. AF, acceleration factor; LR-MRF, low rank MRF; MRF, magnetic resonance fingerprinting; NMSE, normalized mean squared error; SSIM, structural similarity index.

Figure 3 Traditional MRF, LR-MRF and SuperMRF for T2 mapping with # frames reducing from 1,000 to 50 and an AF of 15. Units are in ms. Blue arrows indicate regions of note in the cartilage and meniscus that LR-MRF appreciably differed from reference T2 values. AF, acceleration factor; LR-MRF, low rank MRF; MRF, magnetic resonance fingerprinting; NMSE, normalized mean squared error; SSIM, structural similarity index.

Noise robustness test

Figures 4,5 show the noise robustness test for LR-MRF and SuperMRF. The quantitative T1 and T2 maps estimated by SuperMRF had values near those of the reference maps even at increasing noise. SuperMRF had a consistently higher SSIM and NMSE at each SNR level as compared to LR-MRF, and error maps showed less error in the femoral and tibial cartilage for SuperMRF as compared to LR-MRF for the SNR =10 condition. When comparing SuperMRF and LR-MRF in both T1 and T2 maps at corresponding SNR levels, SuperMRF exhibits lower NMSEs and equivalent-or-greater SSIMs even at low SNR levels for both T1 and T2 maps (Table 4). Quality metrics are preserved for SuperMRF at the lowest SNR level as compared to the LR-MRF reconstruction. M0 maps were not analyzed in this work but are shown in the Supplementary file (Appendix 1).

Figure 4 LR-MRF and SuperMRF noise robustness test for T1 mapping with 200 frames and an AF of 15. Units are in ms. The green arrow indicates the region of muscle that LR-MRF exhibited appreciable heterogeneity and elevation in T1 values as compared to reference. AF, acceleration factor; LR-MRF, low rank magnetic resonance fingerprinting; NMSE, normalized mean squared error; SNR, signal-to-noise ratio; SSIM, structural similarity index.

Figure 5 LR-MRF and SuperMRF noise robustness test for T2 mapping with 200 frames and an AF 15. Units are in ms. The blue arrow indicates a region of cartilage that LR-MRF appreciably differed from reference T2 values. The green arrow indicates the region of muscle that LR-MRF exhibited appreciable heterogeneity in T2 values as compared to reference. AF, acceleration factor; LR-MRF, low rank magnetic resonance fingerprinting; NMSE, normalized mean squared error; SNR, signal-to-noise ratio; SSIM, structural similarity index.

Reconstruction time

SuperMRF training time was ~10 hours, and SuperMRF reconstruction time was ~1 second for a full set of T1, T2, and M0 maps. In contrast, LR-MRF reconstruction required ~30 min processing time for each set of T1, T2, and M0 maps. Conventional MRF reconstruction required <1 second to generate T1, T2, and M0 maps. The MRF reconstruction times did not include dictionary generation, as in practice, the dictionary could be pre-computed and used for subsequent scans for all methods.

Training size

The number of training datasets was gradually reduced to observe its effect on SuperMRF with 200 frames and an AF of 15. As shown in Table 5, both PSNRs and SSIMs remain almost the same, demonstrating the advantage of training using patches.

Prospective real-world test

Figure 6 shows the T1 and T2 maps obtained from conventional MRF, LR-MRF, and SuperMRF using the 500 frame sequence with an AF of 15. The mean values within tissue regions as well as overall are shown in Table 6 with the distributions of values shown in Figure 7. When comparing overall T1 and T2 values between reconstruction methods, two differences were statistically significant: MRF T2 vs. LR-MRF T2 (63.4±14.8 vs. 75.4±17.6, P<0.001) and MRF T2 vs. SuperMRF T2 (63.4±14.8 vs. 72.9±18.4, P=0.002). Using LR-MRF for comparison, absolute and relative differences for MRF and SuperMRF are shown in Table 7.

Figure 6 T1 and T2 maps for the prospective real-world test subjects. Four volunteer subjects scanned on a 3T scanner with 500 frames and fixed k-space AF of 15 are shown. T1 and T2 maps are obtained using conventional MRF, LR-MRF and SuperMRF reconstruction. Residual undersampling artifact is observed in the conventional pattern matching MRF reconstruction, whereas LR-MRF and SuperMRF provide maps without observable undersampling artifact. LR-MRF and SuperMRF exhibit comparable quality. SuperMRF maps were generated using the model from simulation experiments without any additional training. Volunteer subjects (A), (B), and (C) had no known knee pathology, whereas volunteer subject (D) had severe knee-related symptoms and a total knee arthroplasty planned within 6 months. AF, acceleration factor; LR-MRF, low rank MRF; MRF, magnetic resonance fingerprinting.

Figure 7 Violin plots of mean T1 and T2 values in tissue ROIs for the four prospective real-world volunteer subjects shown in Figure 6. Tissue regions include femoral cartilage, tibial cartilage, patellar cartilage, and skeletal muscle. For a given subject and tissue region, the same ROI was used to obtain the mean T1 and T2 values for MRF, LR-MRF, and SuperMRF. Means and standard deviations of tissue regions are reported in Table 6 and differences between MRF and LR-MRF as well as SuperMRF and LR-MRF are reported in Table 7. LR-MRF, low rank MRF; MRF, magnetic resonance fingerprinting; ROI, region of interest.

Discussion

Robustness to frame reduction and noise level

The SSIM for SuperMRF was larger than that of LR-MRF when fewer frames were used as well as at lower SNR. Our interpretation is that the spatial and temporal learning of SuperMRF for temporally-compressed image patches was better able to adapt to the lower quality data in local regions than LR-MRF. While LR-MRF includes a local low rank regularization term in the objective function, image reconstruction in LR-MRF is ultimately global in nature due to the data consistency term, which may be an explanation for the SSIM difference compared to SuperMRF. Further considering the two reconstruction approaches, we did observe differences in the error maps when comparing LR-MRF and SuperMRF. Whereas LR-MRF exhibited a grainy error map that appeared qualitatively as randomly distributed noise in the T1 and T2 maps, SuperMRF appeared to have a greater structure in its error maps. This structured error in SuperMRF was lower overall than the LR-MRF error, and noticeably so in some notable regions, for example, in the femoral and tibial cartilage as observed in the SNR =10 condition for SuperMRF as compared to LR-MRF. However, when looking at other specific regions it is possible that errors were greater when compared to LR-MRF. This property of SuperMRF should be noted when extending its use to specific applications to ensure non-inferior and unbiased performance as compared to LR-MRF. However, similar considerations do exist with LR-MRF, where iterative reconstructions in MRI are known to have image features (e.g., artifacts, noise) that are sensitive to regularization parameters (36,37). Overall, our findings add to the evidence that combined spatial-temporal deep learning reconstruction methods can provide high quality quantitative maps (19,20).

Prospective real-world comparison

In an initial evaluation of SuperMRF for prospective real-world T1 and T2 map reconstruction, overall quality and quantitative values were comparable to that obtained from LR-MRF. Although the overall T1 differences between SuperMRF and LR-MRF were larger than observed for conventional MRF and LR-MRF, the distribution of T1 values was not found to differ statistically between any of the reconstruction methods. Conversely, T2 differences were smaller for SuperMRF and LR-MRF than for conventional MRF and LR-MRF. The observation regarding T2 values is notable, as only conventional MRF T2 was significantly different than LR-MRF or SuperMRF; no significant difference was observed between LR-MRF and SuperMRF. The difference in T2 values for conventional MRF may be due to residual undersampling artifacts that can be observed in the T2 maps, especially in the bone but that may be extending into the cartilage and muscle to a less obvious extent (see Figure 6). These initial observations suggest that SuperMRF may be achieving comparable noise reduction and T1 and T2 quantification as LR-MRF. However, we did observe some qualitative differences between SuperMRF and LR-MRF particularly for the subject with planned total knee arthroplasty especially related to fluid with high T1 and T2 values (see Figure 6D). These pathological features were not present in the training data, which is likely a reason why even with patch-based training that SuperMRF did not capture T1 and T2 values with as high a degree of accuracy as in the other three testing subjects. Further real-world testing is warranted particularly if applying SuperMRF reconstruction to patient data that may not be represented within the training data.

Reconstruction time

The reconstruction by SuperMRF is nearly instantaneous, ~1 second vs. <1 second for conventional MRF dictionary-matching techniques, and much faster than the LR-MRF iterative reconstruction that can take minutes to hours. This rapid reconstruction by SuperMRF is due in part to rapid feedforward processing with GPU but also due to the lack of repeated computations that are used in iterative methods such as LR-MRF. This observed SuperMRF computation time is comparable to a recently reported DL MRF reconstruction method that is capable of generating 120 slices in under 1 minute (38). The only additional computation outside of the network itself is the use of the FFT, which is a standard computation on MR image reconstruction computers, which is fed as input into the network. Unlike iterative reconstructions, this approach reduces the hardware requirements needed and could be rolled out to existing systems after training. However, it should be noted that the LR-MRF implementation in this work could be significantly optimized for computation time by taking advantage of GPU computing, for example as supported by MRIReco (39). Locally low rank compressed sensing has been reported with clinically feasible reconstruction times (~95 s) for a spatial matrix of 288×260×240 with the use of a distributed computing framework (40). While LR-MRF could be supported by efficient implementation or distributed computing, the requirement of substantial or high complexity computational resources is currently a bottleneck to the translation of MRF, perhaps even more so than scanner hardware constraints given that MRF is reportedly feasible on aging MRI scanners (41). Thus, a robust, rapid, deep learning-based reconstruction could offer a step toward clinical translation without the need for intensive computing resources. Even in the case that DL-based reconstruction at the scanner is not as high of quality as LR-MRF types of reconstruction, immediate feedback to the MRI technologist is essential, as motion or other artifacts could require repeated acquisition. This could improve the overall yield of successful MRI exams that use MRF, whereas without immediate reconstruction, patients may need to be called back or readers may need to interpret images with unsatisfactory quality.

Deep learning framework design

While learning the relationship between the undersampled weighted images (implicitly both in k and temporal domain via the loss terms) and the corresponding parameter maps, SuperMRF also exploits the correlation among them. Given this simultaneous learning of spatial and temporal associations, it is anticipated that SuperMRF would be superior in suppressing undersampling artifacts as compared to temporal domain-only, fully connected networks such as DRONE that have not been reported for the acceleration factors evaluated in this work (16). Instead of using fully connected networks, SuperMRF is based on CNN due to its noise robustness nature and its ability to take into account spatial and temporal correlations jointly in a single network, which is advantageous to the two-step fully connected network (20). Furthermore, patch-wise training is utilized in our method. The patch size can not only be optimized empirically to balance the tradeoff between memory usage and data correlation but can also be tuned to adjust the strength of denoising. In general, a larger patch size produces more robustness to noise, but does not necessarily lead to better accuracy overall. So, a larger patch size does not necessarily lead to better performance. The choice of patch size is an important parameter that could be further studied.

Temporal compression of the undersampled MRF frames was observed to improve SuperMRF performance. Instead of inputting all time frames into the network, SuperMRF uses a Bloch equation informed temporal compression in the form of SVD of an MRF dictionary. There are several benefits of doing this. The first is that SVD has the functionality of filtering out noisy information. The second is that temporal SVD compression acts as an efficient dimensionality reduction for the network training problem. That is, temporal SVD compression reduces the total number of model weights that need to be estimated during training. Furthermore, there are significant memory savings. For 1,000 frames, the memory reduction from compression to three singular values in this work was 99.7%, and for 50 frames, the memory reduction is 94%. Relieving computational burden helps the process of the learning and converging of the network, in the meantime, enabling fitting more training data for the networks without increasing the number of GPUs. Note that while the full dictionary was used for SVD-based temporal compression in this work, sparse dictionaries may be sufficient. Despite the superfast reconstruction time of deep learning networks, the training time is always long. Significant memory savings by temporal compression provide the possibility of rapid network training in the future.

MRF acquisition

Although this work focused on the reconstruction problem itself and not on MRF-based mapping of tissue parameters beyond T1, T2, and M0, or sequence optimization, it demonstrated potential to be integrated with advanced MRF data acquisition techniques. MRF has been applied to map many tissue types and magnetic parameters beyond those evaluated in this work including at least T1rho, T2*, fat fraction, diffusion, perfusion, chemical exchange saturation transfer, B0, and B1 (42-48). As such, there are a broad range of sequence possibilities within the MRF framework that can impact the sensitivity to various tissue or magnetic parameters and also affect total scan time. For example, in this study, we did not alter the signal recovery period between repetitions of the MRF acquisition, but as seen in Table 1, this recovery period becomes a significant factor compared to data acquisition time at highly accelerated acquisition. Optimization of MRF data acquisition will also depend on the parameter(s) of interest, as long T1 values may require such long signal recovery periods, therefore limiting the range of sequence possibilities. However, if we are only interested in short T1 values or T2 or T1rho relaxation times, then a short signal recovery period may be reasonable. Such sequence optimization is itself an active area of research, with reported methods including physics-inspired optimization (49) and Cramer-Rao bound optimization (50). Furthermore, non-Cartesian sampling patterns may offer additional improvements in data acquisition efficiency and reconstruction quality as compared to the Cartesian sampling pattern used in this work (51), although gradient system imperfections (52) and off-resonance blurring due to fat must also be considered as part of such sampling trajectories (26). Thus, it is possible that 3D Cartesian and 3D non-Cartesian trajectories may have complementary roles with advantages dependent on the application. Nevertheless, this work demonstrates that SuperMRF can be used as a reconstruction tool for rapid, high quality reconstruction that may be complementary to sequence optimization for rapid total scan times without loss of overall quality. The combination of optimized data acquisition and rapid, robust deep learning reconstruction has the potential to provide high quality tissue parameter maps at significantly reduced scan times relative to conventional tissue parameter mapping methods.

Limitations and future work

This study focused on simulated MRF data with limited testing in prospective real-world MRF scans. The digital phantom testing allowed for well-controlled comparisons between methods and used realistic geometry and variation of T1/T2/M0 in subjects. With this approach, MRF reconstruction could theoretically obtain zero error, therefore any recorded errors are due to limitations of spatial and contrast-weight encoding and recovery from the reconstructions. However, it should be noted that the spatial interpolation of T1/T2/M0 maps and quantization of T1 and T2 values in the simulated knees could introduce correlations that contribute to the observed results. Future work should expand the data used for evaluation and include a wider range of prospectively acquired phantom and in vivo data. To this end, the initial testing of SuperMRF on the same 3 T scanner used in this study demonstrates feasibility on real-world hardware with T1 and T2 map visual quality similar for LR-MRF and SuperMRF, as well as no statistically significant differences observed between LR-MRF and SuperMRF, without any additional training. Note that simulations in this study were performed using single-channel data; the comparisons between SuperMRF and LR-MRF should not be assumed to fully carry over to multi-channel hardware without further evaluation. The technique should also be evaluated for other MRF sequence designs, including different k-space trajectories, acceleration factors, matrix sizes and resolution settings, slice thickness settings, repetition time and flip angle patterns, magnetization preparation pulse schedules, and quantification of additional tissue parameters. Last, we did not investigate possible combinations of SuperMRF with low rank reconstruction. Low rank reconstruction methods continue to be studied and improved results have been reported, for example, to better address measurement errors (53). Low rank and deep learning methods are complementary and may produce improved image quality as compared to either approach independently. Furthermore, the choice of patch size and loss function weights in SuperMRF may be targets for further improving reconstruction quality, drawing upon a hyperparameter optimization strategy as recently reported for MRF (54).

Conclusions

A deep learning reconstruction framework, SuperMRF, is proposed and evaluated for superfast MR fingerprinting reconstruction and aggressive acceleration in both the k-space and temporal directions. With a scan time of fewer than 7 minutes and a reconstruction time of approximately one second, we can obtain a 3D stack of T1, T2, and M0 maps. The proposed SuperMRF outperformed the evaluated reconstruction methods including pattern matching, iterative low rank, and an existing deep learning method, and SuperMRF can achieve robust reconstruction with largely reduced temporal sampling. In the less aggressive acceleration case for prospective real-world comparison, LR-MRF and SuperMRF showed comparable overall quality of the T1 and T2 maps with no statistically significant quantitative differences in overall cartilage and muscle T1 and T2 values. Optimized undersampling masks, a balance between patch size and robustness, and generalizability to different scanners or protocols should be explored in future studies.

Acknowledgments

The authors would like to recognize Fadil Ali (Cleveland Clinic) for his assistance with data analysis.

Footnote

Funding: This work was supported by the National Institutes of Health: National Institute on Aging (grant No. NIH/NIA K25AG070321 to B.L.E.) and National Institute of Arthritis and Musculoskeletal and Skin Diseases (grant No. NIH/NIAMS T32AR007505 to B.L.E., grant No. NIH/NIAMS K25AR078928 to M.Y., and grant No. NIH/NIAMS R01AR077452 to X.L.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-23-1819/coif). D.L., B.L.E., and M.Y. serve as unpaid editorial board members of Quantitative Imaging in Medicine and Surgery. B.L.E. reports that he has received grant support from the National Institutes of Health: National Institute on Aging (grant No. NIH/NIA K25AG070321) and the National Institute of Arthritis and Musculoskeletal and Skin Diseases (grant No. NIH/NIAMS T32AR007505) with payments made to his institution, and that he is a current employee at Clario as of August 2024. M.Y. reports that he has funding support from the National Institute of Arthritis and Musculoskeletal and Skin Diseases (grant No. NIH/NIAMS K25AR078928) with payments made to his institution. H.L. reports that this work was done when he was in State University of New York at Buffalo, and the work has no relation to his current employment at United Imaging Healthcare North America. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Cleveland Clinic Institutional Review Board (Cleveland Clinic IRB number 17-1351, Federal Wide Assurance number FWA00005367), and informed consent was taken from all individual participants.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Ma D, Gulani V, Seiberlich N, Liu K, Sunshine JL, Duerk JL, Griswold MA. Magnetic resonance fingerprinting. Nature 2013;495:187-92. [Crossref] [PubMed]
2. Monga A, Singh D, de Moura HL, Zhang X, Zibetti MVW, Regatte RR. Emerging Trends in Magnetic Resonance Fingerprinting for Quantitative Biomedical Imaging Applications: A Review. Bioengineering (Basel) 2024.
3. Pierre EY, Ma D, Chen Y, Badve C, Griswold MA. Multiscale reconstruction for MR fingerprinting. Magn Reson Med 2016;75:2481-92. [Crossref] [PubMed]
4. Zhao B, Setsompop K, Adalsteinsson E, Gagoski B, Ye H, Ma D, Jiang Y, Ellen Grant P, Griswold MA, Wald LL. Improved magnetic resonance fingerprinting reconstruction with low-rank and subspace modeling. Magn Reson Med 2018;79:933-42. [Crossref] [PubMed]
5. Assländer J, Cloos MA, Knoll F, Sodickson DK, Hennig J, Lattanzi R. Low rank alternating direction method of multipliers reconstruction for MR fingerprinting. Magn Reson Med 2018;79:83-96. [Crossref] [PubMed]
6. Lima da Cruz G, Bustin A, Jaubert O, Schneider T, Botnar RM, Prieto C. Sparsity and locally low rank regularization for MR fingerprinting. Magn Reson Med 2019;81:3530-43. [Crossref] [PubMed]
7. Mazor G, Weizman L, Tal A, Eldar YC. Low-rank magnetic resonance fingerprinting. Med Phys 2018; Epub ahead of print. [Crossref]
8. Yang M, Ma D, Jiang Y, Hamilton J, Seiberlich N, Griswold MA, McGivney D. Low rank approximation methods for MR fingerprinting with large scale dictionaries. Magn Reson Med 2018;79:2392-400. [Crossref] [PubMed]
9. Hamilton JI, Jiang Y, Ma D, Chen Y, Lo WC, Griswold M, Seiberlich N. Simultaneous multislice cardiac magnetic resonance fingerprinting using low rank reconstruction. NMR Biomed 2019;32:e4041. [Crossref] [PubMed]
10. Hu Y, Li P, Chen H, Zou L, Wang H, High-Quality MR. Fingerprinting Reconstruction Using Structured Low-Rank Matrix Completion and Subspace Projection. IEEE Trans Med Imaging 2022;41:1150-64. [Crossref] [PubMed]
11. Liao C, Cao X, Ye H, Chen Y, He H, Chen S, Ding Q, Liu H, Zhong J. Acceleration of MR Fingerprinting with Low Rank and Sparsity Constraint. In: Proc 24th Ann Meet ISMRM; 2016.
12. Doneva M, Amthor T, Koken P, Sommer K, Börnert P. Matrix completion-based reconstruction for undersampled magnetic resonance fingerprinting data. Magn Reson Imaging 2017;41:41-52. [Crossref] [PubMed]
13. Zhao B, Setsompop K, Salat D, Wald LL. Further Development of Subspace Imaging to Magnetic Resonance Fingerprinting: A Low-rank Tensor Approach. Annu Int Conf IEEE Eng Med Biol Soc 2020;2020:1662-6. [Crossref] [PubMed]
14. McGivney DF, Pierre E, Ma D, Jiang Y, Saybasili H, Gulani V, Griswold MA. SVD compression for magnetic resonance fingerprinting in the time domain. IEEE Trans Med Imaging 2014;33:2311-22. [Crossref] [PubMed]
15. Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw 2015;61:85-117. [Crossref] [PubMed]
16. Cohen O, Zhu B, Rosen MS. MR fingerprinting Deep RecOnstruction NEtwork (DRONE). Magn Reson Med 2018;80:885-94. [Crossref] [PubMed]
17. Zhang Q, Su P, Chen Z, Liao Y, Chen S, Guo R, Qi H, Li X, Zhang X, Hu Z, Lu H, Chen H. Deep learning-based MR fingerprinting ASL ReconStruction (DeepMARS). Magn Reson Med 2020;84:1024-34. [Crossref] [PubMed]
18. Hoppe E, Körzdörfer G, Würfl T, Wetzl J, Lugauer F, Pfeuffer J, Maier A. Deep Learning for Magnetic Resonance Fingerprinting: A New Approach for Predicting Quantitative Parameter Values from Time Series. Stud Health Technol Inform 2017;243:202-6.
19. Chen Y, Fang Z, Hung SC, Chang WT, Shen D, Lin W. High-resolution 3D MR Fingerprinting using parallel imaging and deep learning. Neuroimage 2020;206:116329. [Crossref] [PubMed]
20. Fang Z, Chen Y, Liu M, Xiang L, Zhang Q, Wang Q, Lin W, Shen D. Deep Learning for Fast and Spatially-Constrained Tissue Quantification from Highly-Accelerated Data in Magnetic Resonance Fingerprinting. IEEE Trans Med Imaging 2019;38:2364-74. [Crossref] [PubMed]
21. Cohen O, Zhu B, Rosen M. Deep learning for fast MR fingerprinting reconstruction. In: Sci Meet Proc ISMRM; 2017:688.
22. Li H, Yang M, Kim JH, Zhang C, Liu R, Huang P, Liang D, Zhang X, Li X, Ying L. SuperMAP: Deep ultrafast MR relaxometry with joint spatiotemporal undersampling. Magn Reson Med 2023;89:64-76. [Crossref] [PubMed]
23. Li H, Liang Z, Zhang C, Liu R, Li J, Zhang W, Liang D, Shen B, Zhang X, Ge Y, Zhang J, Ying L. SuperDTI: Ultrafast DTI and fiber tractography with deep learning. Magn Reson Med 2021;86:3334-47. [Crossref] [PubMed]
24. Liu J, Saloner D. Accelerated MRI with CIRcular Cartesian UnderSampling (CIRCUS): a variable density Cartesian sampling strategy for compressed sensing and parallel imaging. Quant Imaging Med Surg 2014;4:57-67. [Crossref] [PubMed]
25. Eck BL, Kim J, Yang M, Ma D, Griswold MA, Li X. 3D Cartesian T1rho Magnetic Resonance Fingerprinting Sequence Design for Evaluation of Cartilage and Skeletal Muscle in the Knee. In: Joint Ann Meet ISMRM-ESMRMB ISMRT; 2022.
26. Cencini M, Biagi L, Kaggie JD, Schulte RF, Tosetti M, Buonincontri G. Magnetic resonance fingerprinting with dictionary-based fat and water separation (DBFW MRF): A multi-component approach. Magn Reson Med 2019;81:3032-45. [Crossref] [PubMed]
27. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proc IEEE Conf Comput Vis Pattern Recognit; 2016:770-8.
28. Kim J, Lee JK, Lee KM. Accurate image super-resolution using very deep convolutional networks. In: Proc IEEE Conf Comput Vis Pattern Recognit; 2016:1646-54.
29. Dong C, Loy CC, He K, Tang X. Image Super-Resolution Using Deep Convolutional Networks. IEEE Trans Pattern Anal Mach Intell 2016;38:295-307. [Crossref] [PubMed]
30. Liu F, Feng L, Kijowski R. MANTIS: Model-Augmented Neural neTwork with Incoherent k-space Sampling for efficient MR parameter mapping. Magn Reson Med 2019;82:174-88. [Crossref] [PubMed]
31. Browne MW. Cross-Validation Methods. J Math Psychol 2000;44:108-32. [Crossref] [PubMed]
32. Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press; 2016:801.
33. Vedaldi A, Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Darrell T. Convolutional architecture for fast feature embedding. Cornell University; 2014.
34. Kinga D, Adam JB. A method for stochastic optimization. In: Int Conf Learn Represent; 2015:6.
35. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004;13:600-12. [Crossref] [PubMed]
36. Miao J, Huang F, Narayan S, Wilson DL. A new perceptual difference model for diagnostically relevant quantitative image quality evaluation: a preliminary study. Magn Reson Imaging 2013;31:596-603. [Crossref] [PubMed]
37. Takahashi J, Machida Y, Aoba M, Nawa Y, Kamoshida R, Fukuzawa K, Ohmoto-Sekine Y. Noise power spectrum in compressed sensing magnetic resonance imaging. Radiol Phys Technol 2021;14:93-9. [Crossref] [PubMed]
38. Liu Y, Chen Y, Yap PT. Real-Time Mapping of Tissue Properties for Magnetic Resonance Fingerprinting. Med Image Comput Comput Assist Interv 2021;12906:161-70.
39. Knopp T, Grosser M. MRIReco.jl: An MRI reconstruction framework written in Julia. Magn Reson Med 2021;86:1633-46. [Crossref] [PubMed]
40. Tamir JI, Taviani V, Alley MT, Perkins BC, Hart L, O'Brien K, Wishah F, Sandberg JK, Anderson MJ, Turek JS, Willke TL, Lustig M, Vasanawala SS. Targeted rapid knee MRI exam using T2 shuffling. J Magn Reson Imaging 2019;49:e195-e204. [Crossref] [PubMed]
41. Eck BL, Liu K, Lo WC, Jiang Y, Gulani V, Seiberlich N. Feasibility of Magnetic Resonance Fingerprinting on Aging MRI Hardware. Tomography 2021;8:10-21. [Crossref] [PubMed]
42. Sharafi A, Zibetti MVW, Chang G, Cloos M, Regatte RR. 3D magnetic resonance fingerprinting for rapid simultaneous T1, T2, and T1ρ volumetric mapping of human articular cartilage at 3 T. NMR Biomed 2022;35:e4800. [Crossref] [PubMed]
43. Jaubert O, Arrieta C, Cruz G, Bustin A, Schneider T, Georgiopoulos G, Masci PG, Sing-Long C, Botnar RM, Prieto C. Multi-parametric liver tissue characterization using MR fingerprinting: Simultaneous T(1), T(2), T(2) *, and fat fraction mapping. Magn Reson Med 2020;84:2625-35. [Crossref] [PubMed]
44. Afzali M, Mueller L, Sakaie K, Hu S, Chen Y, Szczepankiewicz F, Griswold MA, Jones DK, Ma D. MR Fingerprinting with b-Tensor Encoding for Simultaneous Quantification of Relaxation and Diffusion in a Single Scan. Magn Reson Med 2022;88:2043-57. [Crossref] [PubMed]
45. Wright KL, Jiang Y, Ma D, Noll DC, Griswold MA, Gulani V, Hernandez-Garcia L. Estimation of perfusion properties with MR Fingerprinting Arterial Spin Labeling. Magn Reson Imaging 2018;50:68-77. [Crossref] [PubMed]
46. Cohen O, Huang S, McMahon MT, Rosen MS, Farrar CT. Rapid and quantitative chemical exchange saturation transfer (CEST) imaging with magnetic resonance fingerprinting (MRF). Magn Reson Med 2018;80:2449-63. [Crossref] [PubMed]
47. Ostenson J, Damon BM, Welch EB. MR fingerprinting with simultaneous T(1), T(2), and fat signal fraction estimation with integrated B(0) correction reduces bias in water T(1) and T(2) estimates. Magn Reson Imaging 2019;60:7-19. [Crossref] [PubMed]
48. Cloos MA, Knoll F, Zhao T, Block KT, Bruno M, Wiggins GC, Sodickson DK. Multiparametric imaging with heterogeneous radiofrequency fields. Nat Commun 2016;7:12445. [Crossref] [PubMed]
49. Jordan SP, Hu S, Rozada I, McGivney DF, Boyacioğlu R, Jacob DC, Huang S, Beverland M, Katzgraber HG, Troyer M, Griswold MA, Ma D. Automated design of pulse sequences for magnetic resonance fingerprinting using physics-inspired optimization. Proc Natl Acad Sci U S A 2021;118:e2020516118. [Crossref] [PubMed]
50. Zhao Bo, Haldar JP, Liao Congyu, Ma Dan, Jiang Yun, Griswold MA, Setsompop K, Wald LL. Optimal Experiment Design for Magnetic Resonance Fingerprinting: Cramér-Rao Bound Meets Spin Dynamics. IEEE Trans Med Imaging 2019;38:844-61. [Crossref] [PubMed]
51. Mehta BB, Ma D, Pierre EY, Jiang Y, Coppo S, Griswold MA. Image reconstruction algorithm for motion insensitive MR Fingerprinting (MRF): MORF. Magn Reson Med 2018;80:2485-500. [Crossref] [PubMed]
52. Speidel T, Meyer CH, Rasche V. Non-cartesian imaging. In: Advances in Magnetic Resonance Technology and Applications Elsevier; 2022:481-98.
53. Qiu Z, Hu S, Zhao W, Sakaie K, Sun JEP, Griswold MA, Jones DK, Ma D. Self-calibrated subspace reconstruction for multidimensional MR fingerprinting for simultaneous relaxation and diffusion quantification. Magn Reson Med 2024;91:1978-93. [Crossref] [PubMed]
54. Cabini RF, Barzaghi L, Cicolari D, Arosio P, Carrazza S, Figini S, Filibian M, Gazzano A, Krause R, Mariani M, Peviani M, Pichiecchio A, Pizzagalli DU, Lascialfari A. Fast deep learning reconstruction techniques for preclinical magnetic resonance fingerprinting. NMR Biomed 2024;37:e5028. [Crossref] [PubMed]