Correctly coding materials and identifying characters marked on materials are very important for steel manufacturing industry to realize informatization management and intelligent manufacturing. However, the steel products manufacturing is often in a high temperature environment, and there are a lot of material storage and retrieval processes, workers are not easily close to the environment and complete tasks, so it is a great challenge to automatically mark and identify characters on the steel products. This paper presents a framework of label characters marking and management for steel materials, furthermore, a kind of marked characters online detection and tracking method has been provided based on machine vision. In addition, some experiments have been done in BaoSteel to mark characters on hot billets and recognize them in multi situations, and the results show that the proposed method is practical, and has provided a helpful exploration in obtaining accurate fundamental data for the intelligent manufacturing system in steelworks.