在現今電力系統的發展中，隨著電力網路規模的擴大以及負載端對電力需求的提升，電力網路越來越複雜的情況下，我們需要一個有效的方式以應付電力網路出現的各種突發狀況。我們提出一個多重智慧型代理人系統，將之應用於電壓的調控上。在我們的系統中，當電壓發生異常時，代理人會偵測並且透過溝通合作的方式來解決電壓異常的問題。此外，代理人透過增強式學習法以得到一個較佳的調控方式。基於分散式的特性，我們將增強式學習法分成了兩個部份。另外，為了增進調控的效能，我們考慮到了調控對於鄰近區域的影響。我們將提出的方法在多次的實驗中進行評估，該方法在多次的實驗中得到驗證。

To protect power systems against different severe disturbances, the ways to effectively control voltage have become the important issue in power systems. In light of that, a Multi-Agent System (MAS) structure has been proposed to deal with the issue of voltage regulation. With our system, once the voltage violations occur, the agents would detect the abnormality and try to eliminate these voltage violations by injecting the reactive power. Besides, to make a good decision under abnormal conditions, a reinforcement learning scheme has been proposed to provide better and faster regulation. Based on the concept of distributed control, there are two parts in our reinforcement learning scheme, self-regulated learning and cooperative learning. On the other hand, an altruistic rate has been proposed to consider the impact of other neighbor agents. The performance of the proposed multi-agent reinforcement learning are demonstrated using various conditions in a benchmark of power networks.

摘要 I
ABSTRACT II
CHAPTER 1 INTRODUCTION 1
1.1 Voltage Control Issue 2
1.2 Reinforcement Learning 4
1.3 Related Researches 7
CHAPTER 2 METHODOLOGY 9
2.1 Agent Design 9
2.1.1 Bus Agent (BA) 10
2.1.2 Control Agent (CA) 10
2.1.3 Environment Agent (EA) 11
2.2 Multi-Agent Reinforcement Learning 11
2.2.1 State and Action Design 12
2.2.2 The Self-regulated Learning 14
2.2.3 The Cooperative Learning 16
2.2.4 Architecture of Proposed Multi-Agent Reinforcement Learning 16
2.3 Process of Multi-Agent System on Voltage Control 18
CHAPTER 3 EXPERIMENT AND SIMULATION 20
3.1 Experimental Parameter in Multi-Agent Reinforcement Learning 21
3.2 Simulation 22
3.3 Case study 24
3.3.1 Case1 25
3.3.2 Case2 26
3.3.3 Case3 28
CHAPTER 4 CONCLUSION 30
REFERENCES 32

[1] Paul, J. P., Leost, J. Y., Tesseron, J. M., “Survey of the secondary voltage control in France: present realization and investigation”, IEEE Transactions on Power Systems, Vol. 2(2), pp. 505-511, 1987.
[2] Ilic-Spong, M., Christensen J., Eichorn K. L., “Secondary voltage control using pilot point information”, IEEE Transactions on Power Systems, Vol. 3(2), pp. 660-688, 1988.
[3] Lefebvre, H., Fragnier, D., Boussion, J. Y., Mallet, P., Bullot, M., “Secondary coordinated voltage control system: feedback of EDF”, Proceedings of the IEEE/PES summer meeting, Vol. 1, pp. 291–295, 2000.
[4] Richard S. Sutton and Andrew G. Barto, “Reinforcement Learning: An introduction”, Cambridge: MIT Press, 1998.
[5] Begovic, M. M., Phadke, A. G., “Control of voltage stability using sensitivity analysis”, IEEE Transactions on Power Systems, Vol. 7(1), pp. 114-123, 1992.
[6] Sheng Gehao, Jiang Xiuceng, Zeng Yi, “Optimal coordination for multi-agent based secondary voltage control in power system”, IEEE PES Transmission and Distribution Conference and Exhibition: Asia and Pacific, 2005.
[7] Baran, M. E., El-Markabi, I. M., “A multiagent-based dispatching scheme for distributed generators for voltage support on distribution feeders”, IEEE Transactions on Power Systems, Vol. 22(1), pp. 52-59, 2007.
[8] Farag, H. E. Z., El-Saadany, E. F., Seethapathy, R., “A two ways communication-based distributed control for voltage regulation in smart distribution feeders”, IEEE Transactions on Smart Grid, Vol. 3(1), pp. 271-281, 2012.
[9] Panasetsky, D. A., Voropai, N. I., “A multi-agent approach to coordination of different emergency control devices against voltage collapse”, IEEE PowerTech international conference, Bucharest, 2009.
[10] Tousi, M. R., Hosseinian, S. H., “A multi-agent-based voltage control in power systems using distributed reinforcement learning”, Transactions of the Society for Modeling and Simulation International, 2011.
[11] Fabio Liuiqi Bellifemine, Giovanni Caire, Dominic Greenwood, “Developing Multi-Agent Systems with JADE”, Wiley, 1 edition, 2007.
[12] MATPOWER, MATLAB Power System Simulation Package, http://www.pserc.cornell.edu/matpower/
[13] Hai Feng Wang, Li, H., Chen, H., “Coordinated secondary voltage control to eliminate voltage violations in power system contingencies”, IEEE Transactions on Power Systems, Vol. 18(2), pp. 588-595, 2003.