Commit 7f829bde authored by Fernand Sieber's avatar Fernand Sieber Committed by Peter Zijlstra
Browse files

sched/core: Optimize core cookie matching check 



Early return true if the core cookie matches. This avoids the SMT mask
loop to check for an idle core, which might be more expensive on wide
platforms.

Signed-off-by: default avatarFernand Sieber <sieberf@amazon.com>
Signed-off-by: default avatarPeter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: default avatarK Prateek Nayak <kprateek.nayak@amd.com>
Reviewed-by: default avatarMadadi Vineeth Reddy <vineethr@linux.ibm.com>
Link: https://patch.msgid.link/20251105152538.470586-1-sieberf@amazon.com
parent 127b9031

kernel/sched/sched.h

+4 −1
Original line number Diff line number Diff line
@@ -1432,6 +1432,9 @@ static inline bool sched_core_cookie_match(struct rq *rq, struct task_struct *p) 
	if (!sched_core_enabled(rq))

		return true;



	if (rq->core->core_cookie == p->core_cookie)

		return true;



	for_each_cpu(cpu, cpu_smt_mask(cpu_of(rq))) {

		if (!available_idle_cpu(cpu)) {

			idle_core = false;
@@ -1443,7 +1446,7 @@ static inline bool sched_core_cookie_match(struct rq *rq, struct task_struct *p) 
	 * A CPU in an idle core is always the best choice for tasks with

	 * cookies.

	 */

	return idle_core || rq->core->core_cookie == p->core_cookie;

	return idle_core;

}



static inline bool sched_group_cookie_match(struct rq *rq,