The cross-domain magnitude interactions have been studied extensively in the past two decades. Majorities of the studies have suggested that the cross-domain magnitude interaction arises from a common magnitude system. However, more recent evidence has argued against a common magnitude system and indicated that the cross-domain magnitude interactions might emerge from cognitive factors like attention and memory. Interestingly, these contradicting pieces of evidence are coming from the studies utilizing only one task-irrelevant magnitude dimension. Thus, it is not easy to evaluate the effect of cognitive factors in a single task-irrelevant magnitude setting. Therefore, to examine the direct influence of attentional mechanism in cross-dimensional magnitude interaction, we manipulated size and number magnitude together to create Size-Number Congruent and Size-Number Incongruent conditions. Further, within congruent and incongruent conditions, we had four different Size-Number combinations such as LargeSize-LargeNumber (LS-LN), SmallSize-SmallNumber (SS-SN), LargeSize-SmallNumber (LS-SN), and SmallSize-LargeNumber (SS-LN). These four conditions were presented to the participants for varying durations. They were asked to judge the duration of the presented Size-Number combinations. Our results indicated that the temporal processing was found to be different for the congruent-large when compared with congruent-small, and no differences were observed when compared to size-number incongruent combinations. Therefore, we argued that the attentional mechanisms modulate the observed differential temporal processing across the different Size-Number combinations.