tournament
- negmas.tournaments.tournament(competitors, config_generator, config_assigner, world_generator, score_calculator, competitor_params=None, n_competitors_per_world=None, round_robin=False, stage_winners_fraction=0.0, agent_names_reveal_type=False, n_agents_per_competitor=1, n_configs=10, max_worlds_per_config=100, n_runs_per_world=5, max_n_configs=None, n_runs_per_config=None, tournament_path=None, total_timeout=None, parallelism='parallel', scheduler_ip=None, scheduler_port=None, tournament_progress_callback=None, world_progress_callback=None, non_competitors=None, non_competitor_params=None, dynamic_non_competitors=None, dynamic_non_competitor_params=None, exclude_competitors_from_reassignment=True, name=None, verbose=False, configs_only=False, compact=False, print_exceptions=True, metric='median', save_video_fraction=0.0, forced_logs_fraction=0.0, video_params=None, video_saver=None, max_attempts=9223372036854775807, extra_scores_to_use=None, **kwargs)[source]
Runs a tournament
- Parameters:
config_generator (
ConfigGenerator
) – Used to generate unique configs that will be used to evaluate competitorsconfig_assigner (
ConfigAssigner
) – Used to generate assignments of competitors to the configs created by theconfig_generator
world_generator (
WorldGenerator
) – A functions to generate worlds for the tournament that follows the assignments made by theconfig_assigner
score_calculator (
Callable
[[list
[World
],dict
[str
,Any
],bool
],WorldRunResults
]) – A function for calculating the score of all agents in a world After it finishes running. The second parameter is a dict describing any scoring context that may have been added by the world config generator or assigneer. The third parameter is a boolean specifying whether this is a dry_run. For dry runs, scores are not expected but names and types should exist in the returnedWorldRunResults
.competitors (
Union
[list
[str
|Agent
],tuple
[str
|Agent
,...
],Sequence
[str
|Agent
]]) – A list of class names for the competitorscompetitor_params (
Optional
[Sequence
[dict
[str
,Any
]]]) – A list of competitor parameters (used to initialize the competitors).n_competitors_per_world (
int
|None
) –The number of competitors allowed in every world. It must be >= 1 and <= len(competitors) or None.
If None or len(competitors), then all competitors will exist in every world.
If 1, then each world will have one competitor
round_robin (
bool
) – Only effective if 1 < n_competitors_per_world < len(competitors). if True, all combinations will be tried otherwise n_competitors_per_world must divide len(competitors) and every competitor appears only in one set.stage_winners_fraction (
float
) – in [0, 1). Fraction of agents to to go to the next stage at every stage. If zero, and round_robin, it becomes a single stage competition.agent_names_reveal_type – If true then the type of an agent should be readable in its name (most likely at its beginning).
n_configs (
int
) – The number of different world configs (up to competitor assignment) to be generated.max_worlds_per_config (
int
) – The maximum number of worlds to run per config. If None, then all possible assignments of competitors within each config will be tried (all permutations).n_runs_per_world (
int
) – Number of runs per world. All of these world runs will have identical competitor assignment and identical world configuration.n_agents_per_competitor – The number of agents of each competing type to be instantiated in the world.
max_n_configs (
int
|None
) – [Depricated] The number of configs to use (it is replaced by separately settingn_config
andmax_worlds_per_config
)n_runs_per_config (
int
|None
) – [Depricated] The number of runs (simulation) for every config. It is replaced byn_runs_per_world
total_timeout (
int
|None
) – Total timeout for the complete processtournament_path (
str
|Path
|None
) – Path at which to store all results. A new folder with the name of the tournament will be created at this path. A scores.csv file will keep the scores and logs folder will keep detailed logsparallelism – Type of parallelism. Can be ‘serial’ for serial, ‘parallel’ for parallel and ‘distributed’ for distributed! For parallel, you can add the fraction of CPUs to use after a colon (e.g. parallel:0.5 to use half of the CPU in the machine). By defaults parallel uses all CPUs in the machine
scheduler_port (
str
|None
) – Port of the dask scheduler if parallelism is dask, dist, or distributedscheduler_ip (
str
|None
) – IP Address of the dask scheduler if parallelism is dask, dist, or distributedworld_progress_callback (
Optional
[Callable
[[World
|None
],None
]]) – A function to be called after every step of every world run (only allowed for serial and parallel evaluation and should be used with cautious).tournament_progress_callback (
Optional
[Callable
[[WorldRunResults
|None
,int
,int
],None
]]) – A function to be called withWorldRunResults
after each world finished processingnon_competitors (
tuple
[str
|Any
] |None
) – A list of agent types that will not be competing but will still exist in the world.non_competitor_params (
tuple
[dict
[str
,Any
]] |None
) – paramters of non competitor agentsdynamic_non_competitors (
tuple
[str
|Any
] |None
) – A list of non-competing agents that are assigned to the simulation dynamically during the creation of the final assignment instead when the configuration is createddynamic_non_competitor_params (
tuple
[dict
[str
,Any
]] |None
) – paramters of dynamic non competitor agentsexclude_competitors_from_reassignment (
bool
) – If true, competitors are excluded from the dyanamic non-competitorsverbose (
bool
) – Verbosityconfigs_only (
bool
) – If true, a config file for eachcompact (
bool
) – If true, compact logs will be created and effort will be made to reduce the memory footprintprint_exceptions (
bool
) – If true, print all exceptions to screenmetric – The metric to use for evaluation
save_video_fraction (
float
) – The fraction of simulations for which to save videosforced_logs_fraction (
float
) – The fraction of simulations for which to always save logs. Notice that this has no effect except if no logs were to be saved otherwise (i.e.no_logs
is passed as True)video_params – The parameters to pass to the video saving function
video_saver – The parameters to pass to the video saving function after the world
max_attempts (
int
) – The maximum number of times to retry running simulationsextra_scores_to_use (
str
|None
) – The type of extra-scores to use. If None normal scores will be used. Only effective if scores is None.kwargs – Arguments to pass to the
config_generator
function
- Return type:
- Returns:
TournamentResults
The results of the tournament or aPath
giving the location where configs were saved