The primary purposes of the study were to see whether incorporating AI technology-based learning tools (hereafter AI) into speaking tasks brings speaking proficiency improvement and to explore whether the nature of tasks in terms of learning principles influences the improvement. The study also examined whether the students’ proficiency levels affect the degree of proficiency improvement. For the above reasons, shadow reading and conversation learning were chosen as two tasks with different learning principles, and for each task, traditional and AI-assisted teaching methods were applied. Eighty students participated in the study, and the results showed that the participants who used AI improved their speaking proficiency overall. However, when considering the nature of the tasks, the degree of proficiency improvement was significantly different depending on the teaching method, in that the conversation learning group showed significantly higher scores than the traditional method group when AI was applied. Regarding proficiency levels, the lower-level students’ proficiency improved significantly in shadow reading. The study results demonstrate that the nature of tasks and learners’ current proficiency level affect the advantages AI could bring to learning English speaking and thus suggest sensible and data-driven incorporation of AI into teaching EFL students.